How to view and increase the default pipe buffer size in Linux (F_GETPIPE_SZ, F_SETPIPE_SZ)

by
 A pipe has a limited capacity.  If the pipe is full, then a write(2) will block or fail. Applications should not rely on a particular capacity: an application should be designed so that a reading process consumes data as soon as it is available, so that a writing process does not remain blocked.

In Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096 bytes on i386). Since Linux 2.6.11, the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with a page size of 4096 bytes). Since Linux 2.6.35, the default pipe capacity is 16 pages, but the capacity can be queried and set using the fcntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ operations.

       F_SETPIPE_SZ (int; since Linux 2.6.35)
              Change the capacity of the pipe referred to by fd to be at least arg bytes.  An unprivileged process can adjust the pipe capacity to any value between the
              system page size and the limit defined in /proc/sys/fs/pipe-max-size (see proc(5)).  Attempts to set the pipe capacity below the page  size  are  silently
              rounded  up  to the page size.  Attempts by an unprivileged process to set the pipe capacity above the limit in /proc/sys/fs/pipe-max-size yield the error
              EPERM; a privileged process (CAP_SYS_RESOURCE) can override the limit.

              When allocating the buffer for the pipe, the kernel may use a capacity larger than arg, if that is convenient for the  implementation.   (In  the  current
              implementation,  the  allocation is the next higher power-of-two page-size multiple of the requested size.)  The actual capacity (in bytes) that is set is
              returned as the function result.

              Attempting to set the pipe capacity smaller than the amount of buffer space currently used to store data produces the error EBUSY.

       F_GETPIPE_SZ (void; since Linux 2.6.35)
              Return (as the function result) the capacity of the pipe referred to by fd.

To validate this let us create a fifo pipe file

# mkfifo /tmp/testfifo

Next below python script (/tmp/change_fifo_size.py) can be used to get the default size and also to increase the pipe buffer size to any new value

#!/usr/bin/env python

import signal
import os
import fcntl
import sys

F_SETPIPE_SZ = 1031  # Linux 2.6.35+
F_GETPIPE_SZ = 1032  # Linux 2.6.35+

def open_fifo(fifo):
    try:
        print "Checking fifo file ..."
        fifo_fd = open(fifo, "rb+")
        print "Pipe size            : "+str(fcntl.fcntl(fifo_fd, F_GETPIPE_SZ))
        fcntl.fcntl(fifo_fd, F_SETPIPE_SZ, 1000000)
        print "Pipe (modified) size : "+str(fcntl.fcntl(fifo_fd, F_GETPIPE_SZ))
        return fifo_fd
    except Exception, e:
        print "Unable to create fifo, error: "+str(e)

fifo_fd = open_fifo("/tmp/testfifo")

Here you can replace the highlighted sections as per your requirement i.e. the location and size of the pipe file

Execute the above script

# /tmp/change_fifo_size.py
Checking fifo file ....
Pipe size            : 65536
Pipe (modified) size : 1048576

So the default fifo file size was 64KB which is increased to 1MB.

I hope the article was useful.