kj's blog

From KernelTrap :

After several days of effort, Linus Torvalds tracked down and posted a patch for a low level data corruption bug [story]. In a series of emails, Linus thoroughly explained the thought process involved in isolating the exact problem. He described the bug as being a difference of expectations between the filesystem and the VM:

“Both filesystem and VM actually _think_ they do the right thing, because they simply have totally different expectations. The filesystem thinks that it should care about dirty buffers (that got marked dirty _after_ they were dirtied), while the filesystem thinks that it cares about dirty _pages_ (that got dirted at any time _before_ ‘writepage()’ was called). Neither is really ‘wrong’, per se, it’s just that the two parts have different expectations, and the _combination_ just doesn’t work. ’set_page_dirty()’ at some point meant ‘the writes have been done’, but these days it really means something else.”

Linus posted the fix in a followup email, “putting on the thinking cap, there’s actually a fairly simple an nonintrusive patch. It still has a tiny tiny race (see the comment), but I bet nobody can really hit it in real life anyway, and I know several ways to fix it, so I’m not really _that_ worried about it.” A comment in the patch explains, “we use this sequence to make sure that (a) we account for dirty stats properly (b) we tell the low-level filesystem to mark the whole page dirty if it was dirty in a pagetable. Only to then (c) clean the page again and return 1 to cause the writeback. This way we avoid all nasty races with the dirty bit in multiple places and clearing them concurrently from different threads.” Another comments explains the remaining race, “if somebody adds the page back to the page tables in between the ‘page_mkclean()’ and the ‘TestClearPageDirty()’, we might have it mapped without the dirty bit set.”

Welcome to Linux.