netatalk.io

search netatalk.io

powered by DuckDuckGo

Dev Docs Network Protocols

Protocol Stack Documentation

Network Protocol Architecture

Netatalk supports two distinct network protocol stacks for AFP communication, allowing it to serve both modern and legacy Mac AFP clients: DSI and AppleTalk, respectively. The former is what the vast majority of cross-platform AFP clients use, while AppleTalk is strictly for communicating with very old Mac clients.

Implementation Files:

AFP over TCP/IP (DSI Protocol)

Data Stream Interface (DSI)

DSI is the session layer protocol that carries AFP over TCP/IP connections. It provides:

Implementation Files:

DSI Message Format

The DSI protocol uses a fixed 16-byte header for all messages:

Implementation Files:

#define DSI_BLOCKSIZ 16
struct dsi_block {
    uint8_t dsi_flags;       /* packet type: request or reply */
    uint8_t dsi_command;     /* command */
    uint16_t dsi_requestID;  /* request ID */
    union {
        uint32_t dsi_code;   /* error code */
        uint32_t dsi_doff;   /* data offset */
    } dsi_data;
    uint32_t dsi_len;        /* total data length */
    uint32_t dsi_reserved;   /* reserved field */
};

Header Layout (Network Byte Order):

DSI Header (16 bytes):
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|FL|CM|     REQUEST ID        | DATA OFFSET/CODE|
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|           TOTAL DATA LENGTH                   |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|                 RESERVED                      |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

DSI Commands

Command Value Description
DSIFUNC_CLOSE 1 Close the DSI session
DSIFUNC_CMD 2 AFP command request
DSIFUNC_STAT 3 Get server status
DSIFUNC_OPEN 4 Open new DSI session
DSIFUNC_TICKLE 5 Keep-alive message
DSIFUNC_WRITE 6 Write data to server
DSIFUNC_ATTN 8 Server attention message

DSI Error Codes

Error Value Description
DSIERR_OK 0x0000 Success
DSIERR_BADVERS 0xfbd6 Bad version
DSIERR_BUFSMALL 0xfbd5 Buffer too small
DSIERR_NOSESS 0xfbd4 No session
DSIERR_NOSERV 0xfbd3 No server
DSIERR_PARM 0xfbd2 Parameter error
DSIERR_SERVBUSY 0xfbd1 Server busy
DSIERR_SESSCLOS 0xfbd0 Session closed
DSIERR_SIZERR 0xfbcf Size error
DSIERR_TOOMANY 0xfbce Too many connections

DSI Session Options

Option Value Description
DSIOPT_SERVQUANT 0x00 Server request quantum
DSIOPT_ATTNQUANT 0x01 Attention quantum
DSIOPT_REPLCSIZE 0x02 AFP replaycache size

Quantum Configuration:

End-to-End Multi-Layer Buffering Architecture

The DSI implementation uses a sophisticated 4-layer buffering system from network to filesystem that is critical for performance and debugging:

Implementation Files:

graph TB
    subgraph "Network Layer"
        A[TCP Socket Buffer<br/>Kernel managed]
        B[Socket Receive Buffer<br/>SO_RCVBUF]
        C[Socket Send Buffer<br/>SO_SNDBUF]
    end

    subgraph "DSI Stream Layer"
        D[DSI Read Buffer<br/>dsi->buffer to dsi->end<br/>Configurable via -dsireadbuf]
        E[DSI Command Buffer<br/>dsi->commands<br/>Receives DSI headers]
        F[DSI Data Buffer<br/>dsi->data array DSI_DATASIZ<br/>64KB reply buffer]
    end

    subgraph "DSI Buffer Management"
        G[Read-ahead Pointers<br/>start, eof, buffer, end]
        H[Quantum Control<br/>server_quantum: 32KB-4GB<br/>Default: 1MB]
        I[Flow Control<br/>dsi_peek and select logic]
    end

    subgraph "AFP Command Layer"
        J[AFP Command Processing<br/>afp_* functions]
        K[Volume Operations<br/>File/Directory operations]
    end

    subgraph "VFS/Filesystem"
        L[VFS Buffer Cache<br/>Kernel page cache]
        M[Disk I/O<br/>Physical storage]
    end

    A --> D
    B --> D
    D --> G
    G --> E
    E --> J
    J --> K
    K --> L
    L --> M

    F --> C
    C --> A

    H -.-> D
    H -.-> F
    I -.-> G

DSI Quantum Management and Flow Control Pipeline

The DSI layer implements sophisticated flow control to prevent buffer overruns and optimize performance:

sequenceDiagram
    participant Client as AFP Client
    participant TCP as TCP Layer
    participant DSI as DSI Stream Layer
    participant Buf as Buffer Manager
    participant AFP as AFP Layer

    Note over Client,AFP: DSI Session Establishment with Quantum Negotiation
    Client->>DSI: OpenSession (request quantum)
    DSI->>Client: Server Quantum: 1MB (default)

    Note over Client,AFP: Large File Transfer with Flow Control
    Client->>TCP: Large AFP Write Request
    TCP->>DSI: Data chunks
    DSI->>Buf: Check buffer space (dsi_peek)

    alt Buffer has space
        Buf->>DSI: Read into dsi->buffer
        DSI->>AFP: Process command
    else Buffer full (dsi->eof >= dsi->end)
        Buf->>DSI: Log "readahead buffer full"
        DSI->>Buf: Wait for space (select)
        Buf->>DSI: Buffer space available
    end

    AFP->>DSI: Response data
    DSI->>Buf: Fill reply buffer (64KB chunks)
    Buf->>TCP: Send via writev() coalescing
    TCP->>Client: Response chunks

    Note over DSI: TCP_NODELAY for small packets
    Note over Buf: Signal blocking during writes

Enhanced DSI State Machine with Buffer Integration

The DSI implementation maintains 9 sophisticated state flags that interact with buffer management:

stateDiagram-v2
    [*] --> Closed

    state "DSI Protocol States" as DSI {
        Closed --> Connected: TCP Connect
        Connected --> SessionOpen: DSI OpenSession
        SessionOpen --> DataReceived: DSI_DATA flag set
        DataReceived --> AFPRunning: DSI_RUNNING flag set
        AFPRunning --> DataReceived: More commands
    }

    state "Buffer Management States" as Buf {
        BufferEmpty --> BufferFilling: dsi_stream_read()
        BufferFilling --> BufferReady: Data available
        BufferReady --> BufferProcessing: AFP command
        BufferProcessing --> BufferEmpty: Command complete
        BufferReady --> BufferFull: dsi->eof >= dsi->end
        BufferFull --> BufferReady: Space available
    }

    state "Advanced Connection States" as Advanced {
        AFPRunning --> Sleeping: DSI_SLEEPING (FPZzz)
        AFPRunning --> ExtSleeping: DSI_EXTSLEEP
        AFPRunning --> NoReply: DSI_NOREPLY (streaming)
        AFPRunning --> Disconnected: DSI_DISCONNECTED
        AFPRunning --> Dying: DSI_DIE (SIGUSR1)
        AFPRunning --> LoggedOut: DSI_AFP_LOGGED_OUT
        AFPRunning --> Reconnecting: DSI_RECONSOCKET
        AFPRunning --> ReconInProgress: DSI_RECONINPROG
    }

    DSI --> Buf: Buffer operations
    DSI --> Advanced: State transitions

    Sleeping --> AFPRunning: Wake up
    ExtSleeping --> AFPRunning: Extended wake
    NoReply --> AFPRunning: Stream complete
    Disconnected --> [*]: Connection lost
    Dying --> [*]: Graceful shutdown
    LoggedOut --> Closed: Client logout
    Reconnecting --> AFPRunning: New socket
    ReconInProgress --> AFPRunning: Reconnect complete

Performance Optimization Architecture

The DSI implementation includes sophisticated optimizations critical for AFP client performance:

graph TB
    subgraph "Network Optimizations"
        A[TCP_NODELAY<br/>Disable Nagle algorithm<br/>Essential for client responsiveness]
        B[writev Coalescing<br/>Combine DSI header + data<br/>Reduce system calls]
        C[Socket Buffer Tuning<br/>SO_RCVBUF/SO_SNDBUF<br/>Match quantum sizes]
    end

    subgraph "DSI Buffer Optimizations"
        D[Read-ahead Buffering<br/>dsireadbuf configuration<br/>Prevents excessive reads]
        E[Quantum Management<br/>1MB default, 32KB-4GB range<br/>Network condition tuning]
        F[Zero-copy Pointers<br/>start/eof/end management<br/>Minimize data copying]
    end

    subgraph "Signal Safety Architecture"
        G[Atomic Write Protection<br/>SIG_BLOCK during transfers<br/>Prevents corruption]
        H[Tickle Suppression<br/>SIGALRM disabled during<br/>large file transfers]
        I[Attention Buffer Isolation<br/>Separate buffers for<br/>SIGHUP/SIGTERM handlers]
    end

    subgraph "AFP Layer Integration"
        J[Streaming Read Pipeline<br/>dsi_readinit/dsi_read<br/>Large file optimization]
        K[Partial Packet Handling<br/>dsi_write incomplete<br/>packet management]
        L[Sendfile Integration<br/>Zero-copy reads when<br/>available if compiled with sendfile support]
    end

    A --> D
    B --> E
    C --> F
    D --> G
    E --> H
    F --> I
    G --> J
    H --> K
    I --> L

Detailed Read/Write Pipeline with Buffer Flow

Shows the complete data path through all buffering layers for performance analysis:

sequenceDiagram
    participant Client as AFP Client
    participant Net as TCP Network
    participant DSI as DSI Layer
    participant Buf as Buffer Manager
    participant Disk as Filesystem

    Note over Client,Disk: Write Pipeline (Client → Server → Disk)

    Client->>Net: AFP Write + Data Stream
    Net->>DSI: TCP segments
    DSI->>Buf: dsi_stream_receive()
    Buf->>DSI: Fill read-ahead buffer (dsireadbuf)
    DSI->>DSI: Parse 16-byte DSI header
    DSI->>Buf: dsi_writeinit(buflen, datasize)

    loop Transfer Loop (until datasize = 0)
        Buf->>DSI: dsi_write() - extract buffered data
        DSI->>Disk: Write data chunk to file
        Disk-->>DSI: Write acknowledgment
        DSI->>Buf: Update remaining datasize counter
    end

    DSI->>Buf: dsi_writeflush() - clear remaining buffers
    DSI->>Net: Send AFP success response
    Net->>Client: Transfer complete

    Note over Client,Disk: Read Pipeline (Server → Client)

    Client->>DSI: AFP Read Request
    DSI->>Disk: Read file data
    Disk-->>DSI: File content
    DSI->>Buf: dsi_readinit() - setup streaming transfer
    DSI->>Net: Send DSI header + initial data

    loop Streaming Loop (quantum-limited)
        DSI->>Buf: dsi_read() - get next chunk
        Buf->>Net: Stream via quantum-sized chunks
        Net->>Client: Data delivery
        DSI->>Buf: Update transfer progress
    end

    DSI->>Buf: dsi_readdone() - cleanup transfer state

TCP Connection Management

sequenceDiagram
    participant Client as AFP Client
    participant Server as afpd

    Client->>Server: TCP SYN (port 548)
    Server-->>Client: TCP SYN-ACK
    Client->>Server: TCP ACK

    Client->>Server: DSI OpenSession
    Server-->>Client: DSI OpenSession Reply

    loop AFP Operations
        Client->>Server: DSI Command (AFP Request)
        Server-->>Client: DSI Command Reply (AFP Response)
    end

    Client->>Server: DSI CloseSession
    Server-->>Client: DSI CloseSession Reply

    Client->>Server: TCP FIN
    Server-->>Client: TCP FIN-ACK

DSI Performance Tuning Guidelines

Critical Buffer Configuration

Read-ahead Buffer Size (-dsireadbuf option):

Server Quantum Configuration:

Implementation Files:

Performance Monitoring

Buffer utilization indicators:

# Monitor DSI buffer warnings in logs
grep "readahead buffer is full" /var/log/afpd.log

# Check quantum negotiation
grep "DSI quantum" /var/log/afpd.log

# Monitor connection state changes
grep "DSI_.*:" /var/log/afpd.log

Key performance metrics:

AFP over AppleTalk (ASP Protocol)

AppleTalk Session Protocol (ASP)

ASP provides session services over the AppleTalk protocol suite:

Implementation Files:

AppleTalk Protocol Stack

Application Layer:     AFP (Apple Filing Protocol)
Session Layer:         ASP (AppleTalk Session Protocol)
Transport Layer:       ATP (AppleTalk Transaction Protocol)
Network Services:      RTMP, NBP, ZIP, AEP
Network Layer:         DDP (Datagram Delivery Protocol)
Data Link Layer:       LocalTalk, EtherTalk, TokenTalk

Implementation Files:

AppleTalk Network Architecture

graph TB
    subgraph "AppleTalk Internetwork"
        subgraph "Network 1000-1000"
            A[AFP Client<br/>1000.10]
            B[Netatalk Server<br/>1000.50]
        end

        subgraph "Network 2000-2000"
            C[AFP Client<br/>2000.25]
            D[LaserWriter<br/>2000.100]
        end

        E[AppleTalk Router]
    end

    A -.-> B
    C -.-> E
    E -.-> B
    C -.-> D

NBP (Name Binding Protocol)

NBP provides name-to-address resolution in AppleTalk networks:

NBP Tuple Format: object:type@zone

Examples:

Implementation Files:

Zone Information Protocol (ZIP)

ZIP manages AppleTalk network zones:

Implementation Files:

AFP Protocol Layer

AFP Commands

AFP defines over 100 commands organized into functional categories:

Implementation Files:

Server Commands

Session Commands

Volume Commands

Directory Commands

File Commands

AFP Data Types

File and Directory IDs

typedef uint32_t AFPDir;     // Directory ID
typedef uint32_t AFPFile;    // File ID
typedef uint16_t AFPFork;    // Fork reference number

Volume and Server Info

struct AFPVolParms {
    uint16_t    vp_attr;        // Volume attributes
    uint16_t    vp_sig;         // Volume signature
    uint32_t    vp_cdate;       // Creation date
    uint32_t    vp_mdate;       // Modification date
    uint32_t    vp_bdate;       // Backup date
    uint16_t    vp_vid;         // Volume ID
    uint32_t    vp_bytes_free;  // Free bytes
    uint32_t    vp_bytes_total; // Total bytes
    uint8_t     vp_name[28];    // Volume name
};

Protocol Implementation Details

DSI Connection Structure

The complete DSI connection structure provides sophisticated connection management:

#define DSI_DATASIZ 65536

typedef struct DSI {
    struct DSI *next;             /* multiple listening addresses */
    AFPObj   *AFPobj;            /* back-reference to AFP object */

    // Server identification
    int      statuslen;
    char     status[1400];       /* server status block */
    char     *signature;         /* server signature */

    // Protocol header and addressing
    struct dsi_block        header;
    struct sockaddr_storage server, client;
    struct itimerval        timer;

    // Connection state management
    int      tickle;            /* tickle count for keepalive */
    int      in_write;          /* write operation in progress */
    int      msg_request;       /* pending message to client */
    int      down_request;      /* pending shutdown request */

    // Performance and flow control
    uint32_t attn_quantum;      /* attention quantum size */
    uint32_t datasize;          /* current data size */
    uint32_t server_quantum;    /* server quantum size */
    uint16_t serverID, clientID; /* connection identifiers */

    // I/O buffers
    uint8_t  *commands;         /* DSI receive buffer */
    uint8_t  data[DSI_DATASIZ]; /* DSI reply buffer (64KB) */
    size_t   datalen, cmdlen;   /* buffer lengths */
    off_t    read_count, write_count; /* I/O statistics */

    // Connection management
    uint32_t flags;             /* DSI state flags */
    int      socket;            /* AFP session socket */
    int      serversock;        /* listening socket (-1 for client connections) */

    // Advanced buffering system
    size_t   dsireadbuf;        /* DSI readahead buffer size */
    char     *buffer;           /* buffer start */
    char     *start;            /* current buffer head */
    char     *eof;              /* end of currently used buffer */
    char     *end;              /* buffer end */

#ifdef USE_ZEROCONF
    char *bonjourname;          /* Bonjour service name */
    int zeroconf_registered;    /* Zeroconf registration status */
#endif

    // Protocol-specific function pointers
    pid_t (*proto_open)(struct DSI *);
    void (*proto_close)(struct DSI *);
} DSI;

DSI State Management

The DSI implementation uses comprehensive state flags:

/* DSI session State flags */
#define DSI_DATA             (1 << 0) /* received a DSI command */
#define DSI_RUNNING          (1 << 1) /* received an AFP command */
#define DSI_SLEEPING         (1 << 2) /* sleeping after FPZzz */
#define DSI_EXTSLEEP         (1 << 3) /* extended sleep mode */
#define DSI_DISCONNECTED     (1 << 4) /* disconnected state after socket error */
#define DSI_DIE              (1 << 5) /* SIGUSR1, shutting down in 5 minutes */
#define DSI_NOREPLY          (1 << 6) /* generate own replies in dsi_write */
#define DSI_RECONSOCKET      (1 << 7) /* new socket from primary reconnect */
#define DSI_RECONINPROG      (1 << 8) /* reconnection in progress */
#define DSI_AFP_LOGGED_OUT   (1 << 9) /* client called afp_logout */

Advanced DSI Features

Readahead Buffering System:

The DSI implementation includes sophisticated buffering for performance: - Configurable readahead buffer size (dsireadbuf) - Smart buffer management with start, eof, and end pointers - Efficient dsi_peek() operations for protocol parsing

Zero-Copy Operations:

#ifdef WITH_SENDFILE
extern ssize_t dsi_stream_read_file(DSI *, int, off_t off, const size_t len, const int err);
#endif

Service Discovery Integration:

Connection Multiplexing:

AppleTalk Address Resolution

// AppleTalk network address
struct at_addr {
    uint16_t s_net;              // Network number
    uint8_t  s_node;             // Node number
};

// NBP name tuple
struct nbptuple {
    char object[32];             // Object name
    char type[32];               // Service type  
    char zone[32];               // Zone name
};

Error Handling

Both protocol stacks implement comprehensive error handling:

AFP Error Codes

DSI Error Recovery

AppleTalk Error Recovery

Performance Characteristics

TCP/IP (DSI) Performance

AppleTalk (ASP) Performance

This protocol documentation provides the foundation for understanding how Netatalk communicates with AFP clients across different network configurations, as well as new and old client systems.

Footnotes

This is a mirror of the Netatalk GitHub Wiki

Last updated 2025-12-27

Valid HTML5 Valid CSS

Netatalk and the Netatalk website are licensed under the GNU General Public License 2.0.