Nodes and Volumes

• 1: Nodes
• 1.1: Node preparation
• 1.2: Node integration flow
• 2: Volumes
• 2.1: Static PV workflow
• 2.2: Volume attributes and mount options

1 - Nodes

1.1 - Node preparation

Install the Lustre client stack

Every node that runs Lustre-backed pods must have:

• mount.lustre and umount.lustre binaries (via lustre-client RPM/DEB).
• Kernel modules compatible with your Lustre servers.
• Network reachability to the Lustre MGS/MDS/OSS endpoints.

Verify installation:

mount.lustre --version
lsmod | grep lustre

Label nodes

The default storage class and daemonset use the label lustre.csi.klustrefs.io/lustre-client=true.

kubectl label nodes <node-name> lustre.csi.klustrefs.io/lustre-client=true

Remove the label when a node no longer has Lustre access:

kubectl label nodes <node-name> lustre.csi.klustrefs.io/lustre-client-

Allow privileged workloads

Klustre CSI pods require:

• privileged: true, allowPrivilegeEscalation: true
• hostPID: true, hostNetwork: true
• HostPath mounts for /var/lib/kubelet, /dev, /sbin, /usr/sbin, /lib, and /lib64

Label the namespace with Pod Security Admission overrides:

kubectl create namespace klustre-system
kubectl label namespace klustre-system \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/audit=privileged \
  pod-security.kubernetes.io/warn=privileged

Maintain consistency

• Keep AMIs or OS images in sync so every node has the same Lustre client version.
• If you use autoscaling groups, bake the client packages into your node image or run a bootstrap script before kubelet starts.
• Automate label management with infrastructure-as-code (e.g., Cluster API, Ansible) so the right nodes receive the lustre-client=true label on join/leave events.

1.2 - Node integration flow

Daemonset host mounts

DaemonSet/klustre-csi-node mounts the following host paths:

• /var/lib/kubelet/plugins and /var/lib/kubelet/pods – required for CSI socket registration and mount propagation.
• /dev – ensures device files (if any) are accessible when mounting Lustre.
• /sbin, /usr/sbin, /lib, /lib64 – expose the host’s Lustre client binaries and libraries to the container.

If your kubelet uses custom directories, update pluginDir and registrationDir in the settings ConfigMap.

CSI socket lifecycle

1. The node plugin listens on csiEndpoint (defaults to /var/lib/kubelet/plugins/lustre.csi.klustrefs.io/csi.sock).
2. The node-driver-registrar sidecar registers that socket with kubelet via registrationDir.
3. Kubelet uses the UNIX socket to call NodePublishVolume and NodeUnpublishVolume when pods mount or unmount PVCs.

If the daemonset does not come up or kubelet cannot reach the socket, run:

kubectl describe daemonset klustre-csi-node -n klustre-system
kubectl logs -n klustre-system daemonset/klustre-csi-node -c klustre-csi

PATH and library overrides

The containers inherit PATH and LD_LIBRARY_PATH values that point at the host bind mounts. If your Lustre client lives elsewhere, override:

• nodePlugin.pathEnv
• nodePlugin.ldLibraryPath

via Helm values or by editing the daemonset manifest.

Health signals

• Kubernetes events referencing lustre.csi.klustrefs.io indicate mount/unmount activity.
• kubectl get pods -n klustre-system -o wide should show one pod per labeled node.
• A missing pod usually means the node label is absent or taints/tolerations are mismatched.

2 - Volumes

2.1 - Static PV workflow

1. Create the PersistentVolume

apiVersion: v1
kind: PersistentVolume
metadata:
  name: lustre-static-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  storageClassName: klustre-csi-static
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: lustre.csi.klustrefs.io
    volumeHandle: lustre-static-pv
    volumeAttributes:
      source: 10.0.0.1@tcp0:/lustre-fs
      mountOptions: flock,user_xattr

• volumeHandle just needs to be unique within the cluster; it is not used by the Lustre backend.
• volumeAttributes.source carries the Lustre management target and filesystem path.

2. Bind with a PersistentVolumeClaim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lustre-static-pvc
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: klustre-csi-static
  volumeName: lustre-static-pv
  resources:
    requests:
      storage: 10Gi

Even though Lustre capacity is managed outside Kubernetes, the storage field should match the PV so the binder succeeds.

3. Mount from workloads

volumes:
  - name: lustre
    persistentVolumeClaim:
      claimName: lustre-static-pvc
containers:
  - name: app
    image: busybox
    volumeMounts:
      - name: lustre
        mountPath: /mnt/lustre

Multiple pods can reference the same PVC because Lustre supports ReadWriteMany. Pods must schedule on labeled nodes (lustre.csi.klustrefs.io/lustre-client=true).

4. Cleanup

Deleting the PVC detaches pods but the PV remains because the reclaim policy is Retain. Manually delete the PV when you no longer need it.

2.2 - Volume attributes and mount options

volumeAttributes

Additional keys (e.g., subdir) can be added in the future; the driver simply passes the map to the Lustre helper script.

Storage class tuning

See the storage class reference for details on:

• allowedTopologies – keep workloads on nodes with the Lustre label.
• reclaimPolicy – typically Retain for static PVs.
• mountOptions – defaults to flock and user_xattr, but you can add noatime, flock, user_xattr, etc.

Override mount options per volume by setting volumeAttributes.mountOptions. This is useful when a subset of workloads needs different locking semantics.

Access modes

• Use ReadWriteMany for shared Lustre volumes.
• ReadOnlyMany is supported when you only need read access.
• ReadWriteOnce offers no benefit with Lustre; prefer RWX.

Lifecycle reminders

• Klustre CSI does not provision or delete Lustre exports. Ensure the server-side directory exists and has the correct permissions.
• Kubernetes capacity values are advisory. Quotas should be enforced on the Lustre server.
• PersistentVolumeReclaimPolicy=Retain keeps PVs around after PVC deletion; clean them up manually to avoid dangling objects.