Cross-platform file locking in Python — fcntl vs msvcrt from scratch

What happens when a GUI app and a separate background process both try to write to the same configuration file at the same time? If the timing is bad, one write clobbers the other, and in the worst case the file ends up corrupted. File locking is the standard answer to this “concurrent writes from multiple processes” problem. Trying to implement it in pure Python across both Unix-like systems and Windows runs straight into a wall: the two platforms expose completely different APIs. This article walks through that difference from the ground up.

Note: “lock” here means inter-process locking — coordinating multiple separate processes on the same machine. That’s different from locking between threads inside a single process (threading.Lock), which is a separate topic.

Why avoid a third-party library

Well-established packages like filelock exist and work fine. But there are situations where adding a dependency is a real cost — packaging a desktop app with PyInstaller or similar, for instance, where every extra dependency adds build complexity and distribution size. Using only the standard library’s fcntl (Unix-like) and msvcrt (Windows) modules, you can implement inter-process locking with zero additional dependencies.

Unix-like systems: fcntl.flock is an advisory lock

The most important property of fcntl.flock() on Unix-like systems (Linux/macOS) is that it’s an advisory lock. The OS kernel tracks the lock state, but it does not forcibly stop a process that ignores the lock and writes anyway. The lock only works if every participating process cooperates and follows the same acquisition protocol.

import fcntl

fd = open('target.lock', 'w')
try:
    # LOCK_EX: exclusive lock / LOCK_NB: non-blocking (fail immediately instead of waiting)
    fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError:
    # another process already holds the lock
    pass

Calling flock() without LOCK_NB blocks in place until the lock becomes available. If a GUI app calls this unguarded on its main thread, the interface can appear to freeze. Most implementations instead try non-blocking acquisition, and if it fails, wait briefly and retry — a polling approach.

Windows: msvcrt.locking locks an explicit byte range

Windows has no fcntl module. The equivalent is msvcrt.locking(), which locks an explicit byte range within a file.

import msvcrt

fd = open('target.lock', 'r+b')
try:
    # LK_NBLCK: attempt a non-blocking lock (over 1 byte)
    msvcrt.locking(fd.fileno(), msvcrt.LK_NBLCK, 1)
except OSError:
    # another process already holds the lock
    pass

Where fcntl.flock() locks the whole file, msvcrt.locking() operates on a specified number of bytes from the current position. When the lock target is a purpose-built sidecar file, as described below, locking just 1 byte is enough — there’s no need to manage complex byte-range bookkeeping.

The core abstraction: branch on sys.platform

Wrapping both APIs into a single class comes down to a simple branch on sys.platform.

import sys

if sys.platform == 'win32':
    import msvcrt
    msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
else:
    import fcntl
    fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)

The imports are placed inside the branch rather than at module top level because import fcntl raises ModuleNotFoundError immediately on Windows. Importing only within the branch that will actually run lets both platform’s code live in the same file without either one breaking the other’s import.

A design choice: don’t lock the target file itself

The naive approach is to open and lock the actual file you’re writing to (say, sites.json) directly. But this runs into a Windows-specific pitfall: when one process has a file open in an exclusive mode, Windows can refuse even basic operations from other processes — including reads or opens in a different mode. Having the locking mechanism itself get in the way of writing the data defeats the purpose.

A common way around this is to introduce a disposable sidecar file, <target-file-name>.lock, and make that the sole object of locking. The actual data file (sites.json) is never opened for locking purposes at all — acquiring and releasing the lock is entirely a matter of creating and deleting sites.json.lock. This separation keeps platform-specific file-open quirks from ever touching the actual data I/O.

Making lock acquisition atomic with O_CREAT | O_EXCL

Creating the sidecar file has its own race condition to worry about. If two processes each perform “check whether the file exists, then create it if not” as two separate steps, another process can slip in between the check and the creation. The fix is the combination of flags passed to os.open(): O_CREAT | O_EXCL.

import os

try:
    fd = os.open('target.lock', os.O_CREAT | os.O_EXCL | os.O_RDWR, 0o644)
    # reaching this line means this process created the file — lock acquired
except FileExistsError:
    # another process already created it — lock acquisition failed
    pass

With O_EXCL set, “create the file if it doesn’t exist, otherwise fail” runs as a single atomic operation inside the OS kernel. There is no window between the check and the creation for another process to interleave, so the race condition is prevented without any additional locking layer.

The remaining gap: stale locks

If the process holding a lock crashes instead of exiting cleanly, the sidecar file can be left behind indefinitely, and no other process will ever be able to acquire the lock again. One practical mitigation is to check the sidecar file’s modification time: if it’s older than some threshold (say, 30 minutes), treat it as stale, delete it, and retry acquisition.

import time

mtime = os.path.getmtime('target.lock')
if time.time() - mtime > 1800:  # older than 30 minutes: treat as stale
    os.remove('target.lock')
    # retry acquisition with O_CREAT | O_EXCL after deleting

This approach isn’t perfect — there’s no way to distinguish “a lock genuinely still held by a process that’s just been running for a long time” from “a lock abandoned by a crash.” In practice, you mitigate this by choosing a threshold generously longer than how long the protected operation normally takes to complete.

Wrapping it as a context manager for with

Bring together everything above — the platform branch, atomic file creation, staleness check, and timeout-with-retry — into a single class, and implement __enter__ / __exit__. Callers then get a clean interface:

with FileLock('sites.json', timeout=10):
    # this block is exclusive across processes
    write_json('sites.json', data)
# lock is released automatically on leaving the block

As long as __exit__ always releases the lock and removes the sidecar file, the lock is guaranteed to be released both on normal completion and when an exception is raised inside the with block — the exact guarantee a context manager exists to provide, without writing try/finally by hand every time.

Summary

fcntl and msvcrt differ in both design philosophy and API shape, but with the surrounding design choices in place — locking a disposable sidecar file rather than the data file itself, and using O_CREAT | O_EXCL for atomic acquisition — the platform branch itself collapses to just a few lines. Worth considering the next time a cross-platform tool needs inter-process locking without pulling in an extra dependency.

The cp932 build-gate crash is the same shape of problem in a different corner: an encoding pitfall that never reproduces during development on macOS and only surfaces on Windows. Both cases come down to the same lesson — platform differences that stay invisible until the code actually runs where the difference matters.