hn.today

Reading and writing are less symmetric than you (probably) think

fgiesen.wordpress.com3 points0 comments
Screenshot of Reading and writing are less symmetric than you (probably) think

Reading and writing appear to be mirror-image operations, but buffering makes their errors behave differently. A read must obtain data before returning it to the program: buffered libraries may request several kilobytes, and the kernel and storage layers fill their buffers before the requested bytes are delivered. If the file is too short or a device or network error occurs, the failure is generally reported during that read. A buffered write, by contrast, may simply copy data into a user-space buffer and report success without making any system call. The actual write - and any errors it encounters - may come later, when the buffer fills or is flushed, repositioned, or closed. Checking only the original write call therefore does not establish that the data was written.

Even a successful system call usually means only that data reached the kernel’s buffer cache, not permanent storage. Programs that need reliable writes must check errors from flushing and closing, and use fsync (or Windows’ FlushFileBuffers) to request that file data reach storage before closing. Errors can still arise late, especially with network filesystems; device or controller caches can also complicate durability. Furthermore, syncing a file may not persist the directory entry that makes it accessible: some systems require syncing the containing directory too. The practical lesson is that reads are comparatively transparent, while durable writes require explicit error checks and a clear understanding of what each layer’s success actually guarantees.

Read on fgiesen.wordpress.com0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Programming

Shaun Thomas: Burnt Toast

Shaun Thomas: Burnt Toast

PostgreSQL’s TOAST storage made the table much smaller, and counting or scanning rows was faster than with the larger uncompressed table. But reading a field inside every toasted value required far more buffer reads, showing that TOAST can trade space savings for slower access to large values. (pgedge.com)

Jepsen: ParticleDB 1.1.4

Jepsen: ParticleDB 1.1.4

Jepsen found 96 bugs in ParticleDB versions 1.1.4 through 1.1.18.11.rc-22, including crashes, data loss, and serious transaction anomalies. Most issues were fixed, but committed writes could still disappear and serializable transactions still had anomalies; Orkes says a later release resolves the remaining problems. (jepsen.io)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.