OSDN Git Service
Darrick J. Wong [Tue, 21 Apr 2015 02:27:19 +0000 (22:27 -0400)]
libext2fs/e2fsck: provide routines to read-ahead metadata
This patch adds to e2fsck the ability to pre-fetch metadata into the
page cache in the hopes of speeding up fsck runs. There are two new
functions -- the first allows a caller to readahead a list of blocks,
and the second is a helper function that uses that first mechanism to
load group data (bitmaps, inode tables).
These new e2fsck routines require the addition of a dblist API to
allow us to iterate a subset of a dblist. This will enable
incremental directory block readahead in e2fsck pass 2.
There's also a function to estimate the readahead given a FS.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 21 Apr 2015 01:48:02 +0000 (21:48 -0400)]
e2fsck: turn inline data symlink into a fast symlink when possible
When there's a problem accessing the EA part of an inline data symlink
and we want to truncate the symlink back to 60 characters (hoping the
user can re-establish the link later on, apparently) be sure to turn
off the inline data flag to convert the symlink back to a regular fast
symlink.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 21 Apr 2015 01:47:18 +0000 (21:47 -0400)]
e2fuzz: fuzz harder
Once we've "fixed" the filesystem, try mounting and modifying it to see
if we can break the kernel.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 12 Apr 2015 22:05:07 +0000 (18:05 -0400)]
Revert "libext2fs: encrypted symlinks are never fast"
This reverts commit
ae73e88e82946595c263e6604f8d53955826ac4f.
The latest kernel patches will now create fast encrypted symlinks
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 12 Apr 2015 12:51:53 +0000 (08:51 -0400)]
Reserve superblock fields s_lpf_ino and s_encryption_level
The s_lpf_ino field is intended to store the location of the lost and
found directory if the root directory becomes encrypted (which is not
yet supported). The s_encryption_level field is designed to allow
support for future changes in the on-disk ext4 encryption format while
this feature under development, without having to burn a large number
of bits in the incompat feature flag.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 12 Apr 2015 12:22:07 +0000 (08:22 -0400)]
Remove compression support
The compression patches were an out-of-kernel patch set that was (a)
only available for ext2, (b) something that was never could be
stablized due to file system corruption, and (c) the most recent
patches were for 3.1, last updated in 2011.
The history of the compression patches has been a bit checkered.
There is a long history here at http://e2compr.sourceforge.net which
lists the perspective of the people working on it from the e2compr
side.
From the ext2/3/4 mainline developers' perspective, initial
compression support was added to e2fsprogs in 2000 (in the Linux 2.2
era), but due to stability concerns the kernel patches were never
merged into the mainline kernel. While there were some sporadic
efforts to try to get the ext2 compression patches working in the 2.4
and 2.6 era, by that time mainline work had moved on to ext4, and the
e2compr approach could only work with 32-bit block numbers and
indirect mapped files.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 6 Apr 2015 00:44:39 +0000 (20:44 -0400)]
Merge branch 'maint' into next
Theodore Ts'o [Mon, 6 Apr 2015 00:43:24 +0000 (20:43 -0400)]
e4crypt: add the get_policy command
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 6 Apr 2015 00:42:58 +0000 (20:42 -0400)]
tune2fs: add ability to enable the encrypt feature
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 6 Apr 2015 00:39:57 +0000 (20:39 -0400)]
Change filename encryption to use CTS mode
Previously we were using a weird hybrid CBC/CTS. Switch things so we
are using straight CTS; this corresponds to changes made in the latest
ext4 encryption patches.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 6 Apr 2015 00:35:50 +0000 (20:35 -0400)]
libext2fs: fix bug in ext2fs_digest_encode()
The ext2fs_digest_encode() function was broken for any input which was
a multiple of 3. Previously we never hit that case, so we never
noticed it was busted. Also fix up the unit test so future problems
like this get noticed quickly.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 30 Mar 2015 18:50:55 +0000 (14:50 -0400)]
Clean up and fix Android build files
Add missing new lib/ext2fs source files that were added for encryption
support. Also move configuration #define's from individual Android.mk
to the android_config.h file, since we've moved away from specifying
configuration #define's on the command-line upstream.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 29 Mar 2015 04:31:52 +0000 (00:31 -0400)]
Update version.h
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 30 Mar 2015 06:11:53 +0000 (02:11 -0400)]
e4crypt: change the UI to use a subcommand style
Also add a new subcommand "new_session", which works much like keyctl
new_session does.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 29 Mar 2015 04:12:53 +0000 (00:12 -0400)]
libext2fs: zero hash in ibody extended attributes
The kernel never updates the extended attribute hash value for
attributes stored in the inode. However, fsck has always checked this
value (if it's nonzero) and will complain if the hash doesn't match
the xattr. Therefore, always zero the hash value when writing to
in-ibody xattrs to avoid creating "corrupt" attribute errors
downstream.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 29 Mar 2015 04:04:46 +0000 (00:04 -0400)]
e2fsck: actually fix inline_data flags problems when user says to do so
fix_problem() returning 1 means to fix the fs error, so do that.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 29 Mar 2015 03:58:20 +0000 (23:58 -0400)]
libext2fs: ext2fs_new_block2() should call alloc_block hook
If ext2fs_new_block2() is called without a specific block map, we
should call the alloc_block hook before checking fs->block_map. This
helps us to avoid a bug in e2fsck where we need to allocate a block
but instead of consulting block_found_map, we use the FS bitmaps,
which (prior to pass 5) could be wrong.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 29 Mar 2015 03:01:08 +0000 (23:01 -0400)]
libext2fs: zero blocks via FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks
Plumb a new call into the IO manager to support translating
ext2fs_zero_blocks calls into the equivalent FALLOC_FL_ZERO_RANGE
fallocate flag primitive when possible. This patch provides _only_
support for file-based images.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 29 Mar 2015 01:39:54 +0000 (21:39 -0400)]
e2fsck: use PROMPT_NONE for FUTURE_SB_LAST_*_FUDGED problems
This allows us to print a message warning the user that there is
something funny going on with their hardware clock (probably time zone
issues caused by trying to be compatible with legacy OS's such as
Windows), without triggering a full file system check.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 29 Mar 2015 00:15:02 +0000 (20:15 -0400)]
Add support for a password salt stored in the superblock
Previously, e4crypt required the user to manually specify the salt
used for their passphrase. This was user unfriendly to say the least.
The e4crypt program can now request the salt using an ioctl, which
will automatically generate the salt if necessary, and keep it in the
ext4 superblock.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Ildar Muslukhov [Sat, 7 Feb 2015 00:30:11 +0000 (16:30 -0800)]
misc: add e4crypt tool
This patch adds new e4crypt tool for encryption management in the ext4
filesystem.
Signed-off-by: Ildar Muslukhov <muslukhovi@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 26 Mar 2015 04:17:48 +0000 (00:17 -0400)]
libext2fs: fix blocksize for SHA512
The blocksize of SHA512 is 128 bytes, not 512.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Ildar Muslukhov [Mon, 2 Mar 2015 16:54:19 +0000 (11:54 -0500)]
misc: teach mke2fs to create encrypted file systems
Also enable support for encryption in e2fsprogs.
Signed-off-by: Ildar Muslukhov <muslukhovi@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 8 Mar 2015 23:09:52 +0000 (19:09 -0400)]
e2fsck: handle encrypted directories which are indexed using htree
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 8 Mar 2015 22:19:05 +0000 (18:19 -0400)]
libext2fs: fix up ext2fs_sha256() and ext2fs_sha512()
Add const annotation to the input pointers; also run the tst_sha256
and tst_sha512 unit tests on a "make check".
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 8 Mar 2015 22:15:47 +0000 (18:15 -0400)]
libext2fs: add ext2fs_digest_encode()
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 8 Mar 2015 22:04:04 +0000 (18:04 -0400)]
debugfs: pretty print encrypted filenames in the ls command
Added the -r (raw) option to print the actual encrypted entry.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 2 Mar 2015 16:40:18 +0000 (11:40 -0500)]
e2fsck: fix spurious duplicate directory entries with encrypted filenames
Use memcmp() instead of strncmp() since encrypted directory names can
contain NUL characters. For non-encrypted directories, we've already
checked for the case of NUL characters in file names, so it's safe to
use memcmp() here in all cases.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 1 Mar 2015 21:58:46 +0000 (16:58 -0500)]
libext2fs: encrypted symlinks are never fast
Teach ext2fs_inodes_has_valid_blocks2() that encrypted symlinks always
use an external block (i.e., we never try to store the symlink in the
i_blocks[] array if it is encrypted).
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 28 Feb 2015 06:09:06 +0000 (01:09 -0500)]
Add files to build on Android
The Android.mk files were taken from the Android AOSP sources, and
updated for the 1.43 next branch. The intention is that this will
allow the repository which is currently located in external/e2fsprogs
with one which is based off of the upstream e2fsprogs. Right now
external/e2fsprogs was not created using "git clone", so it means that
git merges don't work. After the external/e2fsprogs Android
repository is replaced, with one based off the upstream repository,
Android will be able to synchronize with the upstream repository by
pulling and merging from upstream, and then running the script
"./util/gen-android-files" to update any generated files. (This is
necessary because in the Android build system, the Android.mk files
are rather stylized and don't make it easy to run arbitrary shell
scripts during the build phase.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sat, 28 Feb 2015 06:07:34 +0000 (01:07 -0500)]
e2fsck: clean up assertions in dict.c
The C preprocessing symbol NDEBUG is also defined (differently) by
Android's build files, and this was causing compilation failures. So
change assert() to dict_assert() and manually define it instead of
relying on the NDEBUG and <assert.h> semantics.
Also make sure the necessary debugging functions are available is
DICT_NODEBUG is not defined, so that dict.c will correctly build with
and without DICT_NODEBUG.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 24 Feb 2015 04:00:17 +0000 (23:00 -0500)]
libext2fs: make sure dirent functions have prototypes if inline is disabled
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 24 Feb 2015 03:38:46 +0000 (22:38 -0500)]
libext2fs: add functions for sha256 and sha512
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 23 Feb 2015 23:05:21 +0000 (18:05 -0500)]
libe2p: add support for printing and parsing the encryption mode
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 23 Feb 2015 22:44:23 +0000 (17:44 -0500)]
e2fsck: suppress bad name checks for encrypted directories
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 23 Feb 2015 17:54:15 +0000 (12:54 -0500)]
Add support for the read-only feature
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 16 Feb 2015 15:41:20 +0000 (10:41 -0500)]
e2fsck: add a 'yes to all' response in interactive mode
Provide a mechanism for a user to switch fsck into '-y' mode if they
start an interactive session and then get tired of pressing 'y' in
response to numerous prompts.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 16 Feb 2015 15:17:21 +0000 (10:17 -0500)]
Merge branch 'maint' into next
Theodore Ts'o [Fri, 6 Feb 2015 17:46:39 +0000 (12:46 -0500)]
libext2fs: fix potential buffer overflow in closefs()
The bug fix in
f66e6ce4446: "libext2fs: avoid buffer overflow if
s_first_meta_bg is too big" had a typo in the fix for
ext2fs_closefs(). In practice most of the security exposure was from
the openfs path, since this meant if there was a carefully crafted
file system, buffer overrun would be triggered when the file system was
opened.
However, if corrupted file system didn't trip over some corruption
check, and then the file system was modified via tune2fs or debugfs,
such that the superblock was marked dirty and then written out via the
closefs() path, it's possible that the buffer overrun could be
triggered when the file system is closed.
Also clear up a signed vs unsigned warning while we're at it.
Thanks to Nick Kralevich <nnk@google.com> for asking me to look at
compiler warning in the code in question, which led me to notice the
bug in
f66e6ce4446.
Addresses: CVE-2015-1572
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 29 Jan 2015 16:09:07 +0000 (11:09 -0500)]
e2fsck: salvage under-sized dirents by removing them
If the directory processing code ends up pointing to a directory entry
that's so close to the end of the block that there's not even space
for a rec_len/name_len, just substitute dummy values that will force
e2fsck to extend the previous entry to cover the remaining space. We
can't use the helper methods to extract rec_len because that's reading
off the end of the buffer.
This isn't an issue with non-inline directories because the directory
check buffer is zero-extended so that fsck won't blow up.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 28 Jan 2015 16:37:44 +0000 (11:37 -0500)]
e2fsck: improve the inline directory detector
Strengthen the checks that guess if the inode we're looking at is an
inline directory. The current check sweeps up any inline inode if
its length is a multiple of four; now we'll at least try to see if
there's the beginning of a valid directory entry.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 28 Jan 2015 14:00:13 +0000 (09:00 -0500)]
e2fsck: inspect inline dir data as two directory blocks
The design of inline directories (apparently) calls for the i_block[]
region and the EA regions to be treated as if they were two separate
blocks of dirents. Effectively this means that it is impossible for a
directory entry to straddle both areas. e2fsck doesn't enforce this,
so teach it to do so. e2fslib already knows to do this....
Cc: Zheng Liu <gnehzuil.liu@gmail.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 28 Jan 2015 13:53:54 +0000 (08:53 -0500)]
e2fsck: decrement bad count _after_ remapping a duplicate block
Decrement the bad count *after* we've shown that (a) we can allocate a
replacement block and (b) remap the file block. Unfortunately,
the only way to tell if the remapping succeeded is to wait until the
next clone_file_block() call or block_iterate3() returns.
Otherwise, there's a corruption error: we decrease the badcount once in
preparation to remap, then the remap fails (either we can't find a
replacement block or we have to split the extent tree and can't find a
new extent block), so we delete the file, which decreases the badcount
on the block a second time. Later on e2fsck will think that it's
straightened out all the duplicate blocks, which isn't true.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:05:51 +0000 (13:05 -0500)]
e2fsck: handle multiple *ind block collisions with critical metadata
An earlier patch tried to detect indirect blocks that conflicted with
critical FS metadata for the purpose of preventing corrections being
made to those indirect blocks. Unfortunately, that patch cannot
handle more than one conflicting *ind block per file; therefore, use
the ref_block parameter to test the metadata block map to decide if
we need to avoid fixing the *ind block when we're iterating the
block's entries. (We have to iterate the block to capture any blocks
that the block points to, as they could be in use.)
As a side note, in 1B we'll reallocate all those conflicting *ind
blocks and restart fsck, so the contents will be checked eventually.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:12:37 +0000 (13:12 -0500)]
e2fsck: fix message when the journal is deleted and regenerated
When we recreate the journal, don't say that the FS "is now ext3
again", since we could be fixing a damaged ext4 FS journal, which does
not magically convert the FS back to ext3.
[ Use "journaled" instead of "journalled", and also fix the message we
print when deleting the journal --Ted ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 17:32:25 +0000 (12:32 -0500)]
e2fsck: on read error, don't rewrite blocks past the end of the fs
If e2fsck encounters a read error on a block past the end of the
filesystem, don't bother trying to "rewrite" the block. We might
still want to re-try the read to capture FS data marooned past the end
of the filesystem, but in that case e2fsck ought to move the block
back inside the filesystem.
This enables e2fuzz to detect writes past the end of the FS due to
software bugs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:12:12 +0000 (13:12 -0500)]
e2fsck: clear i_block[] when there are too many bad mappings on a special inode
If we decide to clear a special inode because of bad mappings, we need
to zero the i_block array. The clearing routine depends on setting
i_links_count to zero to keep us from re-checking the block maps,
but that field isn't checked for special inodes. Therefore, if we
haven't erased the mappings, check_blocks will restart fsck and fsck
will try to check the blocks again, leading to an infinite loop.
(This seems easy to trigger if the bootloader inode extent map is
corrupted.)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:11:47 +0000 (13:11 -0500)]
tune2fs: direct user to resize2fs for 64bit conversion
If the user tries to enable or disable the 64bit feature via tune2fs,
tell them how to use resize2fs to effect the conversion.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:11:25 +0000 (13:11 -0500)]
tune2fs: abort when trying to enable/disable metadata_csum on mounted fs
Earlier, I tried to make tune2fs abort if the user tried to enable or
disable metadata_csum on a mounted FS, but forgot the exit() call.
Supply it now.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:11:04 +0000 (13:11 -0500)]
tune2fs: disable csum verification before resizing inode
When we're turning on metadata checksumming /and/ resizing the inode
at the same time, disable checksum verification during the
resize_inode() call because the subroutines it calls will try to
verify the checksums (which have not yet been set), causing the
operation to fail unnecessarily.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:10:47 +0000 (13:10 -0500)]
resize2fs: fix regression test to not depend on ext4.ko being loaded
The behavior of the r_fixup_lastbg_big test varies depending on
whether or not ext4.ko is loaded and supports lazy_itable_init. This
makes checking the bg flags after resize2fs hard to predict, so put in
a way to force resize2fs to zero the inode tables, and compare the
output based on lazy_itable_init == 0.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:10:26 +0000 (13:10 -0500)]
libext2fs: fix tdb.c mmap leak
When undoing an expansion of an mmap'd database while cancelling a
transaction, the tdb code prematurely decreases the variable that
tracks the file size, which leads to a region leak during the
subsequent unmap. Fix this by maintaining a separate counter for the
region size.
(This is probably unnecessary since e2undo was the only user of tdb
transactions, but I suppose we could be proactive.)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 18:10:08 +0000 (13:10 -0500)]
libext2fs: strengthen i_extra_isize checks when reading/writing xattrs
Strengthen the i_extra_isize checks to look for obviously too-small
values before trying to operate on inode EAs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 16:02:30 +0000 (11:02 -0500)]
libext2fs: avoid pointless EA block allocation
Use qsort to move the inlinedata attribute to the front of the list
and the empty entries to the end. Then we can use handle->count to
decide if we're done writing xattrs, which helps us to avoid the
situation where we're midway through the attribute list, so we
allocate an EA block to store more, but have no idea that there's
actually nothing left in the list.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 15:59:19 +0000 (10:59 -0500)]
libext2fs: initialize i_extra_isize when writing EAs
If i_extra_isize is zero when we try to write extended attributes,
we'll end up writing the EA magic into the i_extra_isize field, which
causes a subsequent crash on big endian systems (when we try to write
0xEA02 bytes past the inode!). Therefore when the field is zero, set
i_extra_isize to the desired extra_isize size, zero those bytes, and
write the EAs after the end of the extended inode.
v2: Don't bother if we have 128b inodes, and ensure that the value
is 32b-aligned so that the EA magic starts on a 32b boundary.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 15:58:17 +0000 (10:58 -0500)]
debugfs: fix crash in ea_set argument handling
Fix an incorrect check in ea_set that would crash debugfs if someone
runs 'ea_set / foo.bar' (i.e. with no value argument)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 15:56:34 +0000 (10:56 -0500)]
debugfs: document new commands
Document the new journal and xattr commands in the debugfs manpage.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 27 Jan 2015 15:55:05 +0000 (10:55 -0500)]
misc: fix minor testcase problems
Don't write debugfs headers to stdout...
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 20 Jan 2015 23:00:34 +0000 (18:00 -0500)]
Reserve the codepoints for the new INCOMPAT feature ENCRYPT
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Enrico Scholz [Fri, 23 Jan 2015 17:05:17 +0000 (12:05 -0500)]
buildsystem: use 'chmod a-w' instead of 'chmod -w'
'chmod -w' is not portable and can break the build:
| chmod: chmod: ss_err.h: new permissions are r--rw-r--, not r--r--r--
| ss_err.h: new permissions are r--rw-r--, not r--r--r--
| chmod: ss_err.c: new permissions are r--rw-r--, not r--r--r--
| make[2]: *** [ss_err.h] Error 1
This happens because 'chmod -w' is affected by umask. Issue can be
reproduced e.g. by
$ mkdir /tmp/foo
$ setfacl -m d:m:rwx /tmp/foo
$ umask 022
$ touch /tmp/foo/x
$ chmod -w /tmp/foo/x
chmod: /tmp/foo/x: new permissions are r--rw-r--, not r--r--r--
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Justus Winter [Fri, 23 Jan 2015 15:15:57 +0000 (10:15 -0500)]
e2fsck: fix corruption of Hurd filesystems
Previously, e2fsck accessed the field osd2.linux2.l_i_file_acl_high
field without checking that the filesystem is indeed created for
Linux. This lead to e2fsck constantly complaining about certain
nodes:
i_file_acl_hi for inode XXX (/dev/console) is 32, should be zero.
By "correcting" this problem, e2fsck would clobber the field
osd2.hurd2.h_i_mode_high.
Properly guard access to the OS dependent fields.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 19 Jan 2015 21:38:14 +0000 (16:38 -0500)]
e2fuzz: fix clang warning
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 19 Jan 2015 21:37:04 +0000 (16:37 -0500)]
Merge branch 'maint' into next
Darrick J. Wong [Mon, 19 Jan 2015 21:31:49 +0000 (16:31 -0500)]
Fix clang warning and a resource leak
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 13 Jan 2015 00:42:29 +0000 (19:42 -0500)]
e2fsck: close the progress_fd in the logfile child process
If e2fsck.conf's logging feature is enabled, and e2fsck is being run
via systemd-fsck, there will be a deadlock since systemd-fsck is
waiting for progress_fd pipe to be closed, instead of waiting for the
fsck process to exit --- and so the logfile child process won't exit
until it can write out the logfile, and systemd won't continue the
boot process so that the file system can be remounted read-write.
Oops.
Addresses-Debian-Bug: #775234
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 26 Dec 2014 04:43:10 +0000 (23:43 -0500)]
Merge branch 'maint' into next
Conflicts:
lib/ext2fs/inode.c
Theodore Ts'o [Fri, 26 Dec 2014 04:18:32 +0000 (23:18 -0500)]
libext2fs: add sanity check for an invalid itable_used value in inode scan code
If the number of unused inodes is greater than number of inodes a
block group, this can cause an e2fsck -n run of the file system to
crash.
We should add more checks to e2fsck to detect this case directly, but
this will at least protect progams (tune2fs, dump, etc.) which use the
inode_scan abstraction from crashing on an invalid file system.
Addresses-Debian-Bug: #773795
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 17:40:05 +0000 (12:40 -0500)]
tests: test resize2fs 32->64 and 64->32bit conversion code
Add some simple tests to check that flex_bg and meta_bg filesystems
can be converted between 32 and 64bit layouts.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 17:40:03 +0000 (12:40 -0500)]
resize2fs: convert fs to and from 64bit mode
resize2fs does its magic by loading a filesystem, duplicating the
in-memory image of that fs, moving relevant blocks out of the way of
whatever new metadata get created, and finally writing everything back
out to disk. Enabling 64bit mode enlarges the group descriptors,
which makes resize2fs a reasonable vehicle for taking care of the rest
of the bookkeeping requirements, so add to resize2fs the ability to
convert a filesystem to 64bit mode and back.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 17:26:57 +0000 (12:26 -0500)]
libext2fs: speed up the max extent depth api call
The maximum extent tree depth really only depends on the filesystem
block size, so cache the last result if possible.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 15 Dec 2014 03:51:11 +0000 (22:51 -0500)]
Bump version.h for an experimental release
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 03:13:09 +0000 (22:13 -0500)]
resize2fs: don't play stupid games with the block count
While it may be true that playing games with old_fs' block count
during a grow operation shuts up a bunch of warnings, resize2fs
doesn't actually expand the group descriptor array to match the size
we're artificially stuffing into old_fs, which means that if we
actually need to allocate a block out of the larger fs (i.e. we're in
desperation mode), ext2fs_block_alloc_stats2() scribbles on the heap,
leading to crashes if you're lucky and FS corruption if not.
So, rip that piece out and turn off com_err warnings properly and add
a test case to deal with growing a nearly full filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 02:12:37 +0000 (21:12 -0500)]
resize2fs: set bg flags and unused inode count when resizing
Recalculate the unused inode count and the block/inode uninit flags
when resizing a filesystem. This can speed up future e2fsck runs
considerably and will reduce mount times.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 02:10:35 +0000 (21:10 -0500)]
resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride
resize2fs tries to infer the RAID stride by observing differences
between the locations of adjacent block groups' block and inode
bitmaps within the block group. If the two block groups being
compared belong to different flexbgs, however, it'll be fooled by the
large offset into thinking that the FS has an abnormally large RAID
stride.
Therefore, teach it not to get confused by crossing a flexbg.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 15 Dec 2014 02:09:29 +0000 (21:09 -0500)]
resize2fs: use old_fs to detect per-bg metadata blocks to free
When shrinking a filesystem, resize2fs wants to free per-bg metadata
blocks that are no longer needed. This behavior is gated on whether
there's a superblock in the group as told by new_fs. The check really
should be against old_fs, since we're effectively freeing blocks out
of old_fs in the transition to new_fs, but prior to sparse_super2 this
didn't matter since superblocks didn't move, so it didn't matter.
Under sparse_super2, however, there's a superblock in the last group,
so now we need to change the test to use old_fs as it should.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 15 Dec 2014 01:57:09 +0000 (20:57 -0500)]
Merge branch 'maint' into next
Jan Kara [Mon, 15 Dec 2014 01:55:44 +0000 (20:55 -0500)]
badblocks: Limit maximum number of bad blocks
Currently maximum number of bad blocks is not limited in any way.
However our code can really handle at most INT_MAX/2 bad blocks (for
larger numbers binary search indexes start overflowing). So report
number of bad blocks is just too big instead of plain segfaulting.
It won't be too hard to raise the limit but I don't think there's any
real use for disks with over 1 billion of bad blocks...
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Mon, 15 Dec 2014 00:08:59 +0000 (19:08 -0500)]
resize2fs: don't require fsck to print min size
My previous change ended up requiring that the filesystem
be fsck'd after the last mount, even if we are only querying
the minimum size. This is a bit draconian, and it burned
the Fedora installer, which wants to calculate minimum size
for every filesystem in the box at install time, which in turn
requires a full fsck of every filesystem.
Try this one more time, and separate out the tests to make things
a bit more clear. If we're only printing the min size, don't
require the fsck, as this is a bit less dangerous/critical.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 03:08:18 +0000 (22:08 -0500)]
resize2fs: quickly rewrite extent blocks when moving an inode w/ metadata_csum
When we're moving an inode on a metadata_csum filesystem, we need to
rewrite the checksum of all interior nodes of the extent tree. The
current code does this inefficiently via set_bmap, but we can do this
more efficiently through direct iteration of the extent tree.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 03:06:15 +0000 (22:06 -0500)]
resize2fs: don't exit if shrinking sparse_super2 fs to one bg
If we're shrinking a sparse_super2 filesystem to a single block group,
the superblock will be in block 0. This is perfectly valid (for block
group 0 with a blocksize > 1024) so don't exit.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 03:01:15 +0000 (22:01 -0500)]
mke2fs: don't zero inode table blocks that are already zeroed
At mke2fs time, if we discard the device and discard zeroes data,
don't bother zeroing the inode table blocks a second time.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:59:18 +0000 (21:59 -0500)]
tests: testcases for enabling/disabling metadata_csum via tune2fs
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:58:26 +0000 (21:58 -0500)]
tune2fs: enable uninit_bg when disabling metadata_csum
If we're disabling metadata_csum and the user doesn't provide explicit
instructions to enable or disable uninit_bg, assume that they want
uninit_bg to be turned on by default. Otherwise, we lose all block
group flags and unused inode count, which is a big hit to performance.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:57:20 +0000 (21:57 -0500)]
tune2fs: warn if extents are not enabled when turning on metadata_csum
Warn the user if we're trying to enable metadata_csum on a FS that
doesn't support extents (since block maps cannot contain checksums).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:55:22 +0000 (21:55 -0500)]
e2fsck: don't complain about root dir csum failures when getting lnf
Don't complain about checksum failures on the root dir when we're
trying to find l+f if the root dir is going to be rehashed anyway.
The test case for this is t_enable_mcsum in the next patch.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:55:12 +0000 (21:55 -0500)]
e2fsck: only complain about no-checksum directory blocks once
If a directory block lacks space for a checksum and the user directs
e2fsck to fix the directory block (by rehashing it), don't complain a
second time about the checksum verification failure when we get to the
end of the directory block.
Also, don't complain about broken HTREE directories if we're already
planning to rebuild the HTREE directory.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:52:48 +0000 (21:52 -0500)]
dumpe2fs: output cleanup
Don't display unused inodes twice, and make it clear that we're
printing a descriptor checksum.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:50:19 +0000 (21:50 -0500)]
dumpe2fs: reduce dumpe2fs output to 80 columns or less
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:50:13 +0000 (21:50 -0500)]
misc: don't allow mk_hugefiles unless the fs supports extents
The current mk_hugefile code in mke2fs doesn't support creating
non-extent files, so disable the functionality when we're mkfs'ing
without extent support.
The fallocate patches further on will eliminate the need for this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:13:40 +0000 (21:13 -0500)]
libext2fs: add a way to check the theoretical maximum extent tree depth
Add an API so that client programs can discover a reasonable maximum
extent tree depth. This will eventually be used by e2fsck as one of
the criteria to decide if an extent-based file should have its extent
tree rebuilt.
Turn some related magic numbers into constants while we're at it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 01:14:14 +0000 (20:14 -0500)]
libext2fs: set interior tree block goal more intelligently
When we're splitting an extent node, try to allocate the new interior
tree block just prior to the first extent in the block we're trying to
split. The previous logic only set a goal block if we had to split
both the current node and its parent, which is somewhat infrequent.
When that would happen, the goal would start at zero, leading to poor
locality.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 01:00:36 +0000 (20:00 -0500)]
libext2fs: find inode goal when allocating blocks
Try to be a little smarter about where we go to allocate blocks for a
inode. For a given inode and logical offset, set the goal as if the
file were physically continuous. If it's bmapped, just start looking
at wherever lblk 0 is. If that's not possible (the file has no
lblk>pblk mappings, inline data, etc.) then start looking in the
inode's block group.
[ Fixed memory leak --tytso ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 13 Dec 2014 03:12:45 +0000 (22:12 -0500)]
libext2fs: use block_buf in ext2fs_alloc_block2() if it is provided
If the caller supplies a buffer to ext2fs_alloc_block2(), use it
instead of calling ext2fs_zero_blocks2().
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 12 Dec 2014 23:27:12 +0000 (18:27 -0500)]
libext2fs: use a dynamically sized block zeroing buffer
Dynamically grow the block zeroing buffer to a maximum of 4MB, and
allow callers to provide their own zeroed buffer in
ext2fs_zero_blocks2().
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Dmitry Monakhov [Thu, 11 Dec 2014 22:57:12 +0000 (17:57 -0500)]
ext2fs: fix integer overflow in rb_get_bmap_range
bmap_rb_extent is defined as __u64:blk __u64:count. So count can
exceed INT_MAX on populated filesystems.
TESTCASE: xfstest ext4/004
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 11 Dec 2014 22:48:03 +0000 (17:48 -0500)]
e2fsck: force-reread of inode from disk when re-checking a checksum error
When we're rechecking an inode checksum failure, we need to force the
inode to be re-read from disk so that the verification routine runs,
so drop the stashed inode.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 05:01:44 +0000 (00:01 -0500)]
e2fsck: remove 'invalid' wording from error when extent block fails checksum
Don't say the physical block number is invalid if an extent block
fails only the checksum. It passes checks, so it's not invalid.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:57:14 +0000 (22:57 -0500)]
libext2fs: file IO routines should handle uninit blocks
The file IO routines do not handle uninit blocks at all. The read
method should check for the uninit flag and return a buffer of zeroes,
and the write routine should convert unwritten extents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:55:04 +0000 (22:55 -0500)]
libext2fs: refactor extent head creation
Don't open-code the creation of the extent tree header, since
ext2fs_extent_open2() knows how to take care of this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:38:19 +0000 (22:38 -0500)]
tests: regression test for resizing inodes on a metadata_csum fs
Regression test for a problem inadvertently fixed by the patchset
"e2fsprogs/tune2fs: fix memory leak in inode_scan_and_fix()" by Xiaoguang Wang.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>