kevin/linux
1
Fork 0
Code
d20f7043fa
linux/fs
History
Chris Mason d20f7043fa Btrfs: move data checksumming into a dedicated tree 
Btrfs stores checksums for each data block.  Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block.  This means that when we read the inode,
we've probably read in at least some checksums as well.

But, this has a few problems:

* The checksums are indexed by logical offset in the file.  When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data.  It would be faster if we could checksum
the compressed data instead.

* If we implement encryption, we'll be checksumming the plain text and
storing that on disk.  This is significantly less secure.

* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct.  This makes the raid
layer balancing and extent moving much more expensive.

* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.

* There is potentitally one copy of the checksum in each subvolume
referencing an extent.

The solution used here is to store the extent checksums in a dedicated
tree.  This allows us to index the checksums by phyiscal extent
start and length.  It means:

* The checksum is against the data stored on disk, after any compression
or encryption is done.

* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.

This makes compression significantly faster by reducing the amount of
data that needs to be checksummed.  It will also allow much faster
raid management code in general.

The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent.  This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 16:58:54 -05:00
..
9p 9p: fix format warning 2008-10-22 18:48:45 -05:00
adfs vfs: Use const for kernel parser table 2008-10-13 10:10:37 -07:00
affs vfs: Use const for kernel parser table 2008-10-13 10:10:37 -07:00
afs [PATCH] fix ->llseek for more directories 2008-10-23 05:13:21 -04:00
autofs vfs: Use const for kernel parser table 2008-10-13 10:10:37 -07:00
autofs4 autofs4: collect version check return 2008-11-06 15:41:17 -08:00
befs befs: annotate fs32 on tests for superblock endianness 2008-10-16 11:21:46 -07:00
bfs [PATCH] fix ->llseek for more directories 2008-10-23 05:13:21 -04:00
btrfs Btrfs: move data checksumming into a dedicated tree 2008-12-08 16:58:54 -05:00
cifs cifs: fix renaming one hardlink on top of another 2008-11-03 18:31:05 +00:00
coda Switch to a valid email address... 2008-10-27 08:40:17 -07:00
configfs [PATCH] assorted path_lookup() -> kern_path() conversions 2008-10-23 05:12:52 -04:00
cramfs
debugfs integrity: special fs magic 2008-10-13 09:47:43 +11:00
devpts vfs: Use const for kernel parser table 2008-10-13 10:10:37 -07:00
dlm dlm: fix shutdown cleanup 2008-11-13 13:22:34 -06:00
ecryptfs ecryptfs: fix memory corruption when storing crypto info in xattrs 2008-10-30 11:38:46 -07:00
efs [PATCH] switch all filesystems over to d_obtain_alias 2008-10-23 05:13:01 -04:00
exportfs [PATCH] prepare vfs_readdir() callers to returning filldir result 2008-10-23 05:13:10 -04:00
ext2 Merge git://git.kernel.org/pub/scm/linux/kernel/git/viro/bdev 2008-10-23 10:23:07 -07:00
ext3 ext3: Clean up outdated and incorrect comment for ext3_write_super() 2008-11-12 17:17:17 -08:00
ext4 ext4: add checksum calculation when clearing UNINIT flag in ext4_new_inode 2008-11-07 09:21:01 -05:00
fat fat: i_blocks warning fix 2008-11-06 15:41:22 -08:00
freevxfs
fuse saner FASYNC handling on file close 2008-11-01 09:49:46 -07:00
gfs2 [PATCH] switch all filesystems over to d_obtain_alias 2008-10-23 05:13:01 -04:00
hfs [PATCH] move executable checking into ->permission() 2008-10-23 05:13:25 -04:00
hfsplus [PATCH] move executable checking into ->permission() 2008-10-23 05:13:25 -04:00
hostfs [PATCH] introduce fmode_t, do annotations 2008-10-21 07:47:06 -04:00
hpfs [PATCH] hpfs: cleanup ->setattr 2008-10-23 05:12:58 -04:00
hppfs
hugetlbfs vfs: Use const for kernel parser table 2008-10-13 10:10:37 -07:00
isofs [PATCH] switch all filesystems over to d_obtain_alias 2008-10-23 05:13:01 -04:00
jbd jbd: don't give up looking for space so easily in __log_wait_for_space 2008-11-06 22:37:59 -05:00
jbd2 jbd2: don't give up looking for space so easily in __jbd2_log_wait_for_space 2008-11-06 22:38:07 -05:00
jffs2 Merge git://git.infradead.org/mtd-2.6 2008-11-06 15:43:13 -08:00
jfs Merge git://git.kernel.org/pub/scm/linux/kernel/git/viro/bdev 2008-10-23 10:23:07 -07:00
lockd NLM: Set address family before calling nlm_host_rebooted() 2008-10-30 17:19:30 -04:00
minix
ncpfs
nfs NFS: Convert nfs_attr_generation_counter into an atomic_long 2008-10-28 15:21:40 -04:00
nfs_common
nfsd Fix nfsd truncation of readdir results 2008-11-09 15:15:50 -05:00
nls remove CONFIG_KMOD from fs 2008-10-17 02:38:36 +11:00
ntfs [PATCH] switch all filesystems over to d_obtain_alias 2008-10-23 05:13:01 -04:00
ocfs2 ocfs2: Check search result in ocfs2_xattr_block_get() 2008-11-10 09:51:47 -08:00
omfs [PATCH] fix ->llseek for more directories 2008-10-23 05:13:21 -04:00
openpromfs [PATCH] fix ->llseek for more directories 2008-10-23 05:13:21 -04:00
partitions [PATCH] sanitize blkdev_get() and friends 2008-10-21 07:49:06 -04:00
proc Merge branch 'proc-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/adobriyan/proc 2008-11-03 09:59:01 -08:00
qnx4
ramfs Ramfs and Ram Disk pages are unevictable 2008-10-20 08:50:26 -07:00
reiserfs Merge git://git.kernel.org/pub/scm/linux/kernel/git/viro/bdev 2008-10-23 10:23:07 -07:00
romfs
smbfs
sysfs [PATCH] fix ->llseek for more directories 2008-10-23 05:13:21 -04:00
sysv
ubifs Merge branch 'linux-next' of git://git.infradead.org/ubifs-2.6 2008-10-20 09:19:03 -07:00
udf [PATCH] get rid of on-stack dentry in udf 2008-10-23 05:13:15 -04:00
ufs [PATCH] fix ->llseek for more directories 2008-10-23 05:13:21 -04:00
xfs [XFS] XFS: Check for valid transaction headers in recovery 2008-11-10 18:01:50 +11:00
aio.c
anon_inodes.c
attr.c [patch] vfs: make security_inode_setattr() calling consistent 2008-10-23 05:13:27 -04:00
bad_inode.c
binfmt_aout.c
binfmt_elf_fdpic.c binfmt_elf_fdpic: Update for cputime changes. 2008-10-20 20:17:18 -07:00
binfmt_elf.c Merge branch 'v28-timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-10-20 13:19:56 -07:00
binfmt_em86.c Allow recursion in binfmt_script and binfmt_misc 2008-10-16 11:21:38 -07:00
binfmt_flat.c uclinux: fix gzip header parsing in binfmt_flat.c 2008-10-16 11:21:29 -07:00
binfmt_misc.c Allow recursion in binfmt_script and binfmt_misc 2008-10-16 11:21:38 -07:00
binfmt_script.c Allow recursion in binfmt_script and binfmt_misc 2008-10-16 11:21:38 -07:00
binfmt_som.c binfmt_som.c: add MODULE_LICENSE 2008-10-16 11:21:38 -07:00
bio-integrity.c block: Introduce integrity data ownership flag 2008-10-09 08:56:21 +02:00
bio.c block: mark bio_split_pool static 2008-10-09 08:57:05 +02:00
block_dev.c block: fix __blkdev_get() for removable devices 2008-11-06 08:41:56 +01:00
buffer.c fs: buffer lock use lock bitops 2008-10-20 08:52:32 -07:00
char_dev.c [PATCH] tidy up chrdev_open 2008-10-23 05:12:59 -04:00
compat_binfmt_elf.c
compat_ioctl.c
compat.c select: deal with math overflow from borderline valid userland data 2008-10-26 11:22:08 -07:00
dcache.c [PATCH] fs: add a sanity check in d_free 2008-10-23 05:17:12 -04:00
dcookies.c
direct-io.c Remove Andrew Morton's old email accounts 2008-10-16 11:21:32 -07:00
dnotify.c
dquot.c [PATCH] switch quota_on-related stuff to kern_path() 2008-10-23 05:12:44 -04:00
drop_caches.c
eventfd.c
eventpoll.c epoll: avoid double-inserts in case of EFAULT 2008-10-26 12:09:49 -07:00
exec.c coredump: format_corename: don't append .%pid if multi-threaded 2008-10-20 08:52:39 -07:00
fcntl.c
fifo.c [PATCH] introduce fmode_t, do annotations 2008-10-21 07:47:06 -04:00
file_table.c saner FASYNC handling on file close 2008-11-01 09:49:46 -07:00
file.c
filesystems.c proc: move /proc/filesystems to fs/filesystems.c 2008-10-23 14:27:09 +04:00
fs-writeback.c Remove Andrew Morton's old email accounts 2008-10-16 11:21:32 -07:00
generic_acl.c
inode.c
inotify_user.c saner FASYNC handling on file close 2008-11-01 09:49:46 -07:00
inotify.c Fix inotify watch removal/umount races 2008-11-15 12:26:44 -08:00
internal.h
ioctl.c provide generic_block_fiemap() only with BLOCK=y 2008-10-12 11:44:37 -07:00
ioprio.c
Kconfig Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable 2008-11-19 15:59:28 -05:00
Kconfig.binfmt add CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS 2008-10-20 08:52:39 -07:00
libfs.c fs: remove prepare_write/commit_write 2008-10-30 11:38:45 -07:00
locks.c Merge branch 'proc' of git://git.kernel.org/pub/scm/linux/kernel/git/adobriyan/proc 2008-10-23 12:04:37 -07:00
Makefile Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable 2008-11-19 15:59:28 -05:00
mbcache.c
mpage.c Remove Andrew Morton's old email accounts 2008-10-16 11:21:32 -07:00
namei.c [PATCH] move executable checking into ->permission() 2008-10-23 05:13:25 -04:00
namespace.c vfs: fix shrink_submounts 2008-11-12 17:17:17 -08:00
nfsctl.c
no-block.c
open.c [PATCH] introduce fmode_t, do annotations 2008-10-21 07:47:06 -04:00
pipe.c saner FASYNC handling on file close 2008-11-01 09:49:46 -07:00
pnode.c
pnode.h
posix_acl.c
quota_v1.c
quota_v2.c
quota.c
read_write.c [PATCH] generic_file_llseek tidyups 2008-10-23 05:12:59 -04:00
read_write.h
readdir.c [PATCH] prepare vfs_readdir() callers to returning filldir result 2008-10-23 05:13:10 -04:00
select.c select: deal with math overflow from borderline valid userland data 2008-10-26 11:22:08 -07:00
seq_file.c seq_file: add seq_cpumask_list(), seq_nodemask_list() 2008-10-20 08:52:39 -07:00
signalfd.c
splice.c fs: remove prepare_write/commit_write 2008-10-30 11:38:45 -07:00
stack.c
stat.c
super.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/viro/bdev 2008-10-23 10:23:07 -07:00
sync.c
timerfd.c
utimes.c
xattr_acl.c
xattr.c