Dev Docs Architecture Overview
Netatalk Architecture Overview
Process Model
Netatalk uses a three-tier process hierarchy: a controller daemon (netatalk), an AFP parent (afpd), and fork-per-connection worker children (also afpd). When the BDB CNID backend is enabled, an additional cnid_metad → cnid_dbd daemon pair manages persistent file metadata.
Implementation Files:
| Role | File | Description |
|---|---|---|
| Controller | etc/netatalk/netatalk.c |
Service coordinator, libevent loop, child restart |
| AFP Parent | etc/afpd/main.c |
Listener, poll() event loop, fork-per-connection, IPC relay, hint batching |
| Session Fork | libatalk/dsi/dsi_getsess.c |
socketpair() + pipe() creation, fork(), session setup |
| Worker Child | etc/afpd/afp_dsi.c |
afp_over_dsi() — child event loop, AFP command dispatch |
| CNID Metadata | etc/cnid_dbd/cnid_metad.c |
Spawns one cnid_dbd per volume on demand |
| CNID DB Worker | etc/cnid_dbd/main.c |
Per-volume Berkeley DB interface daemon |
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
graph TB
classDef blue fill:#74b9ff,stroke:#0984e3,color:#2d3436,rx:10,ry:10
classDef yellow fill:#ffeaa7,stroke:#fdcb6e,color:#2d3436,rx:10,ry:10
classDef cyan fill:#81ecec,stroke:#00cec9,color:#2d3436,rx:10,ry:10
classDef green fill:#55efc4,stroke:#00b894,color:#2d3436,rx:10,ry:10
classDef salmon fill:#fab1a0,stroke:#e17055,color:#2d3436,rx:10,ry:10
classDef grey fill:#dfe6e9,stroke:#b2bec3,color:#2d3436,rx:10,ry:10
classDef purple fill:#a29bfe,stroke:#6c5ce7,color:#2d3436,rx:10,ry:10
classDef orange fill:#ff9f43,stroke:#e67e22,color:#2d3436,rx:10,ry:10
Client1[AFP Client 1]:::blue
Client2[AFP Client 2]:::blue
ClientN[AFP Client N]:::blue
subgraph "netatalk controller"
NEA["netatalk<br/>libevent loop<br/>fork+exec children<br/>SIGCHLD restart"]:::orange
end
subgraph "afpd parent process"
PARENT["afpd parent<br/>poll() event loop<br/>listener + IPC relay<br/>hint batching (50ms)"]:::yellow
LISTEN["DSI listener sockets<br/>(TCP port 548)"]:::cyan
end
subgraph "afpd worker children (fork-per-connection)"
W1["afpd worker 1<br/>afp_over_dsi()"]:::green
W2["afpd worker 2<br/>afp_over_dsi()"]:::green
WN["afpd worker N<br/>afp_over_dsi()"]:::green
end
subgraph "CNID Services (BDB backend only)"
META["cnid_metad<br/>volume → cnid_dbd mapper"]:::salmon
DBD1["cnid_dbd<br/>(volume 1)"]:::salmon
DBD2["cnid_dbd<br/>(volume 2)"]:::salmon
end
FS["Unix Filesystem<br/>+ Extended Attributes"]:::grey
BDB["Berkeley DB<br/>(per-volume .cnid2/)"]:::grey
NEA -->|"fork+exec"| PARENT
NEA -->|"fork+exec"| META
Client1 -->|"TCP/DSI"| LISTEN
Client2 -->|"TCP/DSI"| LISTEN
ClientN -->|"TCP/DSI"| LISTEN
LISTEN --> PARENT
PARENT -->|"fork via<br/>dsi_getsession()"| W1
PARENT -->|"fork"| W2
PARENT -->|"fork"| WN
W1 -->|"IPC socketpair"| PARENT
W2 -->|"IPC socketpair"| PARENT
WN -->|"IPC socketpair"| PARENT
PARENT -.->|"hint pipe<br/>(parent→child)"| W1
PARENT -.->|"hint pipe"| W2
PARENT -.->|"hint pipe"| WN
W1 --> FS
W2 --> FS
WN --> FS
W1 -->|"TCP"| META
W2 -->|"TCP"| META
META -->|"UNIX socket"| DBD1
META -->|"UNIX socket"| DBD2
DBD1 --> BDB
DBD2 --> BDB
Controller Daemon: netatalk
netatalk.c is the top-level service coordinator. It uses libevent (event_base_new(), event_base_dispatch()) to handle signals and a 1-second timer. On startup it:
- Parses configuration via
afp_config_parse() - Calls
run_process()tofork()+execv()theafpdparent and optionallycnid_metad - Optionally starts a D-Bus session daemon and Spotlight indexer
- Registers with Zeroconf (Avahi/mDNSResponder) via
zeroconf_register() - Enters the libevent dispatch loop
The timer_cb() callback checks every second whether afpd or cnid_metad have died and restarts them. sigchld_cb() reaps exited children via waitpid(). sigterm_cb() initiates graceful shutdown with a KILL_GRACETIME (5 second) timeout before sending SIGKILL.
AFP Parent: afpd Listener
etc/afpd/main.c runs the AFP parent process. After configuration parsing and signal setup, it enters the main poll() event loop (inside main()):
while (1) {
poll_timeout = (hint_buf_count() > 0) ? HINT_FLUSH_INTERVAL_MS : -1;
ret = poll(asev->fdset, asev->used, poll_timeout);
// 1. Process SIGCHLD — reap dead children
// 2. Error handling (EINTR → continue)
// 3. Config reload on SIGHUP
// 4. Process fd events — LISTEN_FD → dsi_start(), IPC_FD → ipc_server_read()
// 5. Flush buffered hints when 50ms timeout or buffer full
}
The parent multiplexes two fd types via struct asev (defined in include/atalk/util.h):
- LISTEN_FD: DSI server sockets — on activity, calls dsi_start() which invokes dsi_getsession() to fork a worker child
- IPC_FD: Per-child IPC socketpairs — on activity, calls ipc_server_read() to process child messages
The poll timeout is dynamically set: -1 (block indefinitely) when no hints are buffered, or 50ms when hints need flushing. After processing fd events, hint_flush_pending() writes batched cache invalidation hints to all sibling children.
Session Forking: dsi_getsession()
dsi_getsession() (in libatalk/dsi/dsi_getsess.c) creates the communication channels and forks the worker:
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
graph TB
classDef green fill:#55efc4,stroke:#00b894,color:#2d3436,rx:10,ry:10
classDef yellow fill:#ffeaa7,stroke:#fdcb6e,color:#2d3436,rx:10,ry:10
classDef purple fill:#a29bfe,stroke:#6c5ce7,color:#2d3436,rx:10,ry:10
classDef grey fill:#dfe6e9,stroke:#b2bec3,color:#2d3436,rx:10,ry:10
classDef cream fill:#f8f4e8,stroke:#d4c5a9,color:#2d3436,rx:10,ry:10
classDef cyan fill:#81ecec,stroke:#00cec9,color:#2d3436,rx:10,ry:10
SP["socketpair(PF_UNIX, SOCK_STREAM)<br/>→ ipc_fds[0], ipc_fds[1]"]:::purple
PIPE["pipe()<br/>→ hint_pipe[0] (read), hint_pipe[1] (write)"]:::purple
FORK["dsi->proto_open(dsi)<br/>fork()"]:::cream
subgraph "Parent (returns afp_child_t*)"
P_CLOSE["close(ipc_fds[1]), close(hint_pipe[0])"]:::grey
P_ADD["server_child_add(pid, ipc_fds[0], hint_pipe[1])"]:::yellow
end
subgraph "Child (returns NULL → afp_over_dsi)"
C_CLOSE["close(ipc_fds[0]), close(hint_pipe[1])"]:::grey
C_STORE["obj->ipc_fd = ipc_fds[1]<br/>obj->hint_fd = hint_pipe[0]"]:::green
C_SESS{"DSIFUNC_STAT<br/>or OPEN?"}:::cream
C_STAT["Send status, exit"]:::grey
C_OPEN["dsi_opensession()<br/>→ afp_over_dsi()"]:::green
end
SP --> FORK
PIPE --> FORK
FORK -->|"parent path"| P_CLOSE --> P_ADD
FORK -->|"child path"| C_CLOSE --> C_STORE --> C_SESS
C_SESS -->|"STAT"| C_STAT
C_SESS -->|"OPEN"| C_OPEN
Worker Child: afp_over_dsi()
afp_over_dsi() (in etc/afpd/afp_dsi.c) is the child’s main event loop. It:
- Initializes the directory cache via
dircache_init() - Sets up signal handlers (SIGTERM, SIGHUP, SIGURG for reconnect, SIGALRM for tickle)
- Enters a dual
poll()loop monitoring both the client DSI socket and the hint pipe from the parent - When client data arrives, calls
dsi_stream_receive()to read the DSI frame - Dispatches AFP commands via the
afp_switch[]function table - Between commands, calls
process_cache_hints()to apply cross-process dircache invalidation hints received via the hint pipe
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
graph TB
classDef blue fill:#74b9ff,stroke:#0984e3,color:#2d3436,rx:10,ry:10
classDef yellow fill:#ffeaa7,stroke:#fdcb6e,color:#2d3436,rx:10,ry:10
classDef green fill:#55efc4,stroke:#00b894,color:#2d3436,rx:10,ry:10
classDef purple fill:#a29bfe,stroke:#6c5ce7,color:#2d3436,rx:10,ry:10
classDef grey fill:#dfe6e9,stroke:#b2bec3,color:#2d3436,rx:10,ry:10
classDef cream fill:#f8f4e8,stroke:#d4c5a9,color:#2d3436,rx:10,ry:10
START["afp_over_dsi() entry<br/>dircache_init(), signal setup"]:::green
subgraph "Inner poll() loop"
BUFCHK{"DSI buffer<br/>has data?"}:::cream
STCHK{"Sleep/disconnect<br/>state?"}:::cream
POLL["poll(dsi_socket, hint_pipe, -1)<br/>block until event"]:::purple
HINTS1["process_cache_hints()"]:::yellow
PERR{"poll error?"}:::cream
HPIPE["Hint pipe ready →<br/>process_cache_hints()"]:::yellow
CDATA["Client socket POLLIN"]:::blue
end
RECV["dsi_stream_receive()"]:::blue
DISPATCH{"DSI command<br/>type?"}:::cream
CMD["AFP command dispatch<br/>afp_switch[function](obj, ...)"]:::green
REPLY["dsi_cmdreply(dsi, err)"]:::green
TICK["DSI tickle handling"]:::grey
CLOSE["afp_dsi_close() + exit"]:::grey
START --> BUFCHK
BUFCHK -->|"data ready"| RECV
BUFCHK -->|"empty"| STCHK
STCHK -->|"special state"| HINTS1 --> RECV
STCHK -->|"normal"| POLL
POLL --> PERR
PERR -->|"EINTR"| BUFCHK
PERR -->|"OK"| HPIPE
HPIPE --> CDATA
CDATA -->|"POLLIN"| RECV
RECV --> DISPATCH
DISPATCH -->|"DSIFUNC_CMD"| CMD --> REPLY --> BUFCHK
DISPATCH -->|"DSIFUNC_TICKLE"| TICK --> BUFCHK
DISPATCH -->|"DSIFUNC_CLOSE"| CLOSE
Cache Architecture
Full details: Caching Architecture
Each worker child maintains a private, per-process three-tier cache hierarchy (no shared memory between workers):
| Tier | What | Eviction | Key Files |
|---|---|---|---|
| Dircache | struct dir entries (files & dirs) with cached stat fields |
ARC (adaptive replacement) or LRU | dircache.c, dircache.h |
| AD cache | AppleDouble metadata (FinderInfo, dates, attributes) inline in struct dir |
Invalidated by ctime change | ad_cache.c |
| Rfork cache | Resource fork content buffers | Dedicated LRU with configurable byte budget | ad_cache.c |
The ARC dircache (default 64K entries) uses four lists — T1 (recent), T2 (frequent), B1/B2 (ghost lists) — to self-tune the balance between recency and frequency. Ghost entries retain full data for zero-allocation promotion. LRU mode is available as a fallback.
Cross-process coherence is maintained by an IPC cache hint relay: when a child modifies the filesystem, it sends a hint (REFRESH, DELETE, or DELETE_CHILDREN) to the parent via the IPC socketpair. The parent batches hints with 50ms coalescing and writes them to sibling children via dedicated per-child hint pipes. Receivers apply invalidation via dir_modify() or dir_remove() against their local dircache.
IPC System
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
graph LR
classDef green fill:#55efc4,stroke:#00b894,color:#2d3436,rx:10,ry:10
classDef yellow fill:#ffeaa7,stroke:#fdcb6e,color:#2d3436,rx:10,ry:10
classDef purple fill:#a29bfe,stroke:#6c5ce7,color:#2d3436,rx:10,ry:10
classDef cyan fill:#81ecec,stroke:#00cec9,color:#2d3436,rx:10,ry:10
subgraph "Worker Child"
IPC_W["IPC socketpair<br/>(obj->ipc_fd)"]:::purple
HINT_R["Hint pipe read<br/>(obj->hint_fd)"]:::cyan
end
subgraph "afpd Parent"
IPC_P["IPC socketpair<br/>(afpch_ipc_fd)"]:::purple
HINT_W["Hint pipe write<br/>(afpch_hint_fd)"]:::cyan
BUF["hint_buf[128]<br/>50ms batching"]:::yellow
end
IPC_W <-->|"session transfer<br/>state updates<br/>cache hints<br/>login done"| IPC_P
IPC_P -->|"ipc_relay_cache_hint()"| BUF
BUF -->|"hint_flush_pending()"| HINT_W
HINT_W -->|"22-byte messages<br/>PIPE_BUF atomic"| HINT_R
Each worker child has two communication channels to the parent (defined in include/atalk/server_ipc.h):
| Channel | Type | Direction | Purpose |
|---|---|---|---|
| IPC socketpair | socketpair(PF_UNIX, SOCK_STREAM) |
Bidirectional | Session transfer, state updates, cache hints (child→parent) |
| Hint pipe | pipe() |
Parent→Child | Cache invalidation hint delivery (22-byte messages, PIPE_BUF atomic) |
IPC commands (IPC_DISCOLDSESSION=0, IPC_GETSESSION=1, IPC_STATE=2, IPC_VOLUMES=3, IPC_LOGINDONE=4, IPC_CACHE_HINT=5). Wire format: 14-byte header + variable payload.
Child Process Table
The parent maintains a hash table of all active children via server_child_t (defined in include/atalk/server_child.h):
typedef struct {
pthread_mutex_t servch_lock;
int servch_count; // Current active session count
int servch_nsessions; // Max allowed sessions
afp_child_t *servch_table[CHILD_HASHSIZE]; // Hash table (size 32)
} server_child_t;
Each child is tracked by afp_child_t:
typedef struct afp_child {
pid_t afpch_pid; // Worker process PID
uid_t afpch_uid; // Connected user's UID
int afpch_ipc_fd; // IPC socketpair (parent end)
int afpch_hint_fd; // Hint pipe write end (parent→child)
int16_t afpch_state; // AFP session state
char *afpch_volumes; // Mounted volumes string
// ... client identification fields for session transfer
} afp_child_t;
Hash function: HASH(pid) = ((pid >> 8) ^ pid) & (CHILD_HASHSIZE - 1) — defined in libatalk/util/server_child.c.
Core Data Structures
struct dir — Dircache Entry
Defined in include/atalk/directory.h. Caches both files (DIRF_ISFILE) and directories. Fields are grouped by alignment for cache locality:
| Group | Fields | Purpose |
|---|---|---|
| Pointers (8B) | d_fullpath, d_m_name, d_u_name, d_m_name_ucs2, qidx_node |
Path and name strings, queue position |
| Rfork cache | dcache_rfork_buf, rfork_lru_node |
Tier 2 resource fork content |
| Time/Size (8B) | dcache_ctime, dcache_ino, dcache_mtime, dcache_size, dcache_rlen |
Cached stat + AD metadata |
| IDs (4B) | d_flags, d_pdid, d_did, d_vid, d_offcnt |
CNID, volume, parent IDs |
| Stat cache | dcache_mode, dcache_uid, dcache_gid |
Cached permissions |
| ARC metadata | arc_list (uint8_t) |
ARC list membership: T1/T2/B1/B2/NONE |
| AD cache (bytes) | dcache_afpfilei[4], dcache_finderinfo[32], dcache_filedatesi[16] |
Cached AppleDouble metadata |
struct ofork — Open Fork
Defined in etc/afpd/fork.h. Tracks open data/resource forks per session:
struct ofork {
struct file_key key; // {dev, inode} for identity
struct adouble *of_ad; // AppleDouble handle
struct vol *of_vol; // Volume pointer
cnid_t of_did; // Directory ID
uint16_t of_refnum; // AFP fork reference number
int of_flags; // AFPFORK_DATA, AFPFORK_RSRC, etc.
struct ofork **prevp, *next; // Hash chain
};
Fork flags (in etc/afpd/fork.h): AFPFORK_DATA, AFPFORK_RSRC, AFPFORK_META, AFPFORK_DIRTY, AFPFORK_ACCRD, AFPFORK_ACCWR, AFPFORK_MODIFIED, AFPFORK_VIRTUAL.
struct adouble — AppleDouble Handle
Defined in include/atalk/adouble.h. Manages the dual-fork file format that stores Mac-specific metadata alongside Unix files:
- AD Version EA (
AD_VERSION_EA= 0x00020002): Modern — stores metadata in extended attributes - AD Version 2 (
AD_VERSION2= 0x00020000): Legacy — uses._filenamesidecar files - Entry IDs:
ADEID_FINDERI(9),ADEID_FILEDATESI(8),ADEID_AFPFILEI(14),ADEID_RFORK(2), etc.
CNID Backends
CNID (Catalog Node ID) provides persistent, stable file identifiers across renames and moves. Netatalk supports three backends via a pluggable module system (libatalk/cnid/cnid.c):
| Backend | Type | Files | Description |
|---|---|---|---|
| dbd (Berkeley DB) | External daemon | cnid_dbd.c, main.c, cnid_metad.c |
Default. cnid_metad spawns one cnid_dbd per volume. Provides crash recovery, transactions, concurrent access. |
| sqlite | Embedded | cnid_sqlite.c |
Lightweight embedded option. No separate daemon needed. |
| mysql | Embedded client | cnid_mysql.c |
Connects to external MySQL/MariaDB server. Suitable for clustered deployments. |
BDB Backend Architecture
The BDB backend uses a two-process design:
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
graph LR
classDef blue fill:#74b9ff,stroke:#0984e3,color:#2d3436,rx:10,ry:10
classDef salmon fill:#fab1a0,stroke:#e17055,color:#2d3436,rx:10,ry:10
classDef grey fill:#dfe6e9,stroke:#b2bec3,color:#2d3436,rx:10,ry:10
classDef green fill:#55efc4,stroke:#00b894,color:#2d3436,rx:10,ry:10
W1["afpd worker"]:::green
W2["afpd worker"]:::green
META["cnid_metad<br/>TCP listener<br/>spawns cnid_dbd per volume"]:::salmon
subgraph "Per-Volume cnid_dbd"
DBD1["cnid_dbd (vol1)<br/>Berkeley DB interface"]:::salmon
DBD2["cnid_dbd (vol2)<br/>Berkeley DB interface"]:::salmon
end
DB1["BDB .cnid2/<br/>cnid2.db<br/>devino.db<br/>didname.db"]:::grey
DB2["BDB .cnid2/<br/>cnid2.db<br/>devino.db<br/>didname.db"]:::grey
W1 -->|"TCP socket"| META
W2 -->|"TCP socket"| META
META -->|"UNIX domain socket<br/>+ fd passing"| DBD1
META -->|"UNIX domain socket<br/>+ fd passing"| DBD2
DBD1 --> DB1
DBD2 --> DB2
Connection flow (from etc/cnid_dbd/cnid_metad.c):
1. afpd worker connects to cnid_metad via TCP
2. cnid_metad passes the client fd to the appropriate cnid_dbd via a UNIX domain socket
3. afpd then communicates directly with cnid_dbd via the passed TCP socket
Simplified Architecture (Embedded CNID)
With SQLite or MySQL, no separate CNID daemon is needed:
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
graph TB
classDef blue fill:#74b9ff,stroke:#0984e3,color:#2d3436,rx:10,ry:10
classDef yellow fill:#ffeaa7,stroke:#fdcb6e,color:#2d3436,rx:10,ry:10
classDef green fill:#55efc4,stroke:#00b894,color:#2d3436,rx:10,ry:10
classDef grey fill:#dfe6e9,stroke:#b2bec3,color:#2d3436,rx:10,ry:10
classDef orange fill:#ff9f43,stroke:#e67e22,color:#2d3436,rx:10,ry:10
Client["AFP Client"]:::blue
subgraph "netatalk"
NEA["netatalk controller"]:::orange
end
subgraph "afpd"
PARENT["afpd parent"]:::yellow
W["afpd workers"]:::green
end
subgraph "Storage"
SQLITE["SQLite / MySQL"]:::grey
FS["Unix Filesystem<br/>+ Extended Attributes"]:::grey
end
Client -->|"TCP/DSI"| PARENT
NEA -->|"fork+exec"| PARENT
PARENT -->|"fork"| W
W --> SQLITE
W --> FS
Data Flow: AFP File Operation
%%{ init: { 'themeVariables': { 'fontSize': '14px' }, 'flowchart': { 'nodeSpacing': 15, 'rankSpacing': 40 } } }%%
sequenceDiagram
participant C as AFP Client
participant W as afpd Worker
participant DC as Dircache<br/>(ARC + AD + Rfork)
participant CNID as CNID Backend
participant FS as Filesystem
C->>W: FPGetFileDirParms (DSI frame)
W->>DC: dircache_search_by_did(vol, cnid)
alt Cache hit (T1/T2)
DC-->>W: struct dir (cached stat + AD)
W->>DC: ad_metadata_cached() → ad_rebuild_from_cache()
DC-->>W: struct adouble (from cache)
else Cache miss
W->>CNID: cnid_resolve(did) → path
CNID-->>W: unix path
W->>FS: stat(path)
FS-->>W: struct stat
W->>FS: ad_metadata() → getxattr()
FS-->>W: AppleDouble data
W->>DC: dircache_add() + ad_store_to_cache()
end
W-->>C: AFP reply (file parameters)
Note over W: If file was modified:
W->>W: ipc_send_cache_hint(vid, cnid, REFRESH)
W-->>W: → afpd parent → hint_flush_pending() → sibling workers
Signal Handling
| Signal | Controller (netatalk) |
AFP Parent (afpd main) |
Worker Child (afp_dsi.c) |
|---|---|---|---|
| SIGTERM | sigterm_cb(): graceful shutdown, 5s timeout |
afp_goaway(SIGTERM): server_child_kill(SIGTERM), _exit(0) |
afp_dsi_die(): send attention, close session |
| SIGHUP | sighup_cb(): reload volumes, re-register Zeroconf |
Sets reloadconfig=1 → re-read afp.conf |
afp_dsi_reload(): sets reload_request=1 |
| SIGCHLD | sigchld_cb(): waitpid(), check shutdown |
Sets gotsigchld=1 → child_handler() |
child_handler(): wait(NULL) |
| SIGURG | — | — | afp_dsi_transfer_session(): primary reconnect |
| SIGUSR1 | — | nologin++, disallow new logins |
afp_dsi_timedown(): 5-minute shutdown timer |
| SIGALRM | — | — | alarm_handler(): tickle/timeout/disconnect |
| SIGINT | — | — | afp_dsi_debug(): toggle max_debug logging + cache dump |
Footnotes
This is a mirror of the Netatalk GitHub Wiki
Last updated 2026-04-06