Dev Docs Data Flow

Data Flow Architecture

Data Flow Architecture

Audited against Netatalk d63155e13 on 2026-10-10. This describes the current source checkout; older releases may differ.

Request path

flowchart LR
    client[AFP client] --> dsi[DSI framing]
    dsi --> dispatch[afp_switch command handler]
    dispatch --> path[Volume and path resolution]
    path --> cache[Directory and metadata caches]
    path --> cnid[SQLite or MySQL CNID]
    dispatch --> ad[AppleDouble and VFS]
    ad --> fs[Filesystem]
    dispatch --> reply[DSI reply or streamed data]
    reply --> client

The controller does not relay AFP file data. The session worker reads the client socket, invokes an AFP command handler, and sends the reply directly. Persistent CNID queries are library calls in that worker, with an external database connection only for MySQL.

Connection and authentication

The parent’s dsi_start() in etc/afpd/main.c calls dsi_getsession() in libatalk/dsi/dsi_getsess.c.
Session creation sets up the IPC socketpair and hint pipe; the TCP transport accepts and forks through libatalk/dsi/dsi_tcp.c.
Status requests are answered without a logged-in AFP session. OpenSession negotiates DSI options in libatalk/dsi/dsi_opensess.c.
The worker enters afp_over_dsi() in etc/afpd/afp_dsi.c, initializes caches and the idle worker, and reads framed requests through dsi_stream_receive().
Before authentication, afp_switch points at preauth_switch in etc/afpd/switch.c. afp_login()/afp_login_ext() in etc/afpd/auth.c select a UAM. Successful login establishes Unix identity and enables the authenticated command table.

The worker polls a client socket, hint pipe, and signal self-pipe. A request already buffered by DSI can bypass poll(). Deferred signals are processed in main-loop context, including reconnect socket transfer, shutdown, and diagnostics.

Volume mount and path resolution

load_afp_conf_vols() and creatvol() in libatalk/util/netatalk_conf.c create authorized volume definitions. afp_openvol() in etc/afpd/volume.c initializes runtime state, including the CNID handle, root directory, and optional virtual icon/search state.

struct vol is configuration plus runtime state; struct dir holds cached filesystem identity, names, stat fields, and metadata; struct path carries the result and stat status for an operation. Definitions are in include/atalk/volume.h and include/atalk/directory.h.

cname() and dirlookup() in etc/afpd/directory.c resolve AFP paths and IDs. Cache lookups may validate against filesystem stat information, accelerated by pfd_ostat() in etc/afpd/pfd_cache.c. A CNID backend failure must remain distinguishable from an absent file.

Filename conversion uses libatalk/unicode/charcnv.c and the selected volume/Mac charset. AFP 3.x UTF-8 names and older Mac names take different conversion paths. Do not treat charset selection or Unicode normalization as merely a throughput setting.

Read and write

afp_openfork() in etc/afpd/fork.c checks access and deny modes, opens the selected fork, and allocates an ofork through etc/afpd/ofork.c. Multiple forks of one file can share an AppleDouble handle.

Operation	Main path
Read	`afp_read()` / `afp_read_ext()` → open-fork lookup and lock checks → resource-fork cache, `ad_read()`, or eligible sendfile → DSI streaming
Write	`afp_write()` / `afp_write_ext()` → validate fork and offset → consume DSIWrite data → `ad_write()` or eligible recvfile → mark fork changes and invalidate cached metadata
Flush / close	Fork and volume handlers flush metadata/data as required, release locks and descriptors, and emit notifications

DSI receive readahead buffers incoming network data. It is not disk read-ahead for outgoing file reads. Sendfile and recvfile are optional paths with platform and storage restrictions; a configuration switch does not guarantee every transfer uses them.

libatalk/adouble/ad_recvfile.c tracks both consumed socket bytes and persisted file bytes. On a failed write it discards already-consumed pipe data and drains the remaining DSI payload to preserve frame alignment. Later AFP requests must not be parsed from leftover write data. See test/afpd/subtests_recvfile.c.

strict locking controls temporary POSIX byte-range locks around data I/O. It does not remove the explicit AFP byte-range-lock commands or the AFP deny-mode machinery. See libatalk/adouble/ad_lock.c and Shared Volumes.

AppleDouble metadata and resource forks

include/atalk/adouble.h defines struct adouble and entry IDs. The data fork is the ordinary Unix file. AppleDouble stores FinderInfo, AFP attributes, dates, CNID hints, and resource-fork information.

Storage depends on volume mode and platform:

Legacy v2 uses .AppleDouble/<name> metadata/resource-fork files.
EA mode stores Netatalk metadata in xattrs; resource-fork storage is platform-dependent. On macOS it uses native resource-fork access; other builds can use ._<name> storage.
Generic AFP extended attributes are routed through the selected VFS EA operations; they are distinct from the AppleDouble metadata format.

Path/open behavior is in libatalk/adouble/ad_open.c, persistence in libatalk/adouble/ad_flush.c, fork I/O in libatalk/adouble/ad_read.c and libatalk/adouble/ad_write.c, and VFS selection in libatalk/vfs/vfs.c.

ad_metadata_cached() in etc/afpd/ad_cache.c reconstructs read-only metadata from struct dir when valid, or reads storage and populates the cache. A caller needing writable descriptors must open them explicitly.

Enumeration, mutation, and notifications

etc/afpd/enumerate.c walks directories and constructs requested AFP parameter bitmaps. Metadata can be served from the directory and AD caches. The virtual Icon\r entry is appended when enabled at the volume root and no real icon file takes precedence.

Create, delete, rename, and move handlers update filesystem and CNID state and invalidate their local caches. They can also produce:

Notification	Destination and purpose
Ordinary cache hint	Child → parent IPC → sibling hint pipes; best-effort cache repair
CNID reset hint	Matching sessions are terminated so clients discard stale IDs
FCE	Optional external UDP listeners/scripts
Spotlight index event	Selected backend’s indexing hook, where implemented

These mechanisms have separate consumers and failure behavior. FCE is not the internal cache-coherence channel. A shared CNID database is not shared process memory.

Spotlight Search

etc/afpd/spotlight.c handles Spotlight RPCs; etc/afpd/spotlight_marshalling.c and libatalk/dalloc/dalloc.c encode/decode the request objects. Queries dispatch through sl_backend_ops in include/atalk/spotlight.h.

Backend	Source	Query/index model
`cnid` (default)	etc/spotlight/cnid/sl_cnid.c	Searches known CNID names with scope and filesystem/access filtering
`localsearch`	etc/spotlight/localsearch/sl_localsearch.c	Queries the external Localsearch index using SPARQL
`xapian`	etc/spotlight/xapian/sl_xapian.c, etc/spotlight/xapian/xapian_wrapper.cc	Embedded index, reconciliation, and incremental update hooks

Search candidates are resolved to current paths/CNIDs and checked before returning file metadata. spotlight results limit bounds results across backends. CNID search sees database-known entries and is not a general full-content filesystem index.

Error and session recovery

AFP errors are defined in include/atalk/afp.h. Transport errors, filesystem errors, and CNID errors are mapped at their respective callers; they do not share one universal recovery routine.

Primary reconnect locates a session by random token and matching UID in libatalk/util/server_child.c, passes the new socket through IPC, and wakes the old worker via SIGURG/self-pipe processing. A database reset ends the session instead of allowing continued use of old IDs.

See CNID Database, Network Protocols, and Troubleshooting.

Footnotes

Generated from the Netatalk GitHub Wiki

Last updated 2026-10-21