OSDN Git Service
Chao Yu [Tue, 17 Oct 2017 09:33:41 +0000 (17:33 +0800)]
f2fs: trace f2fs_lookup
This patch adds trace for f2fs_lookup.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Weichao Guo [Sat, 14 Oct 2017 00:13:32 +0000 (08:13 +0800)]
f2fs: skip searching non-exist range in truncate_hole
Let's skip entire non-exist area to speed up truncate_hole by
using get_next_page_offset.
Signed-off-by: Weichao Guo <guoweichao@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 13 Oct 2017 17:27:45 +0000 (10:27 -0700)]
f2fs: expose some sectors to user in inline data or dentry case
If there's some data written through inline data or dentry, we need to shouw
st_blocks. This fixes reporting zero blocks even though there is small written
data.
Cc: stable@vger.kernel.org
Reviewed-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: avoid link file for quotacheck]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 13 Oct 2017 02:12:53 +0000 (19:12 -0700)]
f2fs: avoid stale fi->gdirty_list pointer
When doing fault injection test, f2fs_evict_inode() didn't remove gdirty_list
which incurs a kernel panic due to wrong pointer access.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 7 Oct 2017 07:08:05 +0000 (00:08 -0700)]
f2fs/crypto: drop crypto key at evict_inode only
This patch avoids dropping crypto key in f2fs_drop_inode, so we can guarantee
it happens only at evict_inode.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 9 Oct 2017 09:55:19 +0000 (17:55 +0800)]
f2fs: fix to avoid race when accessing last_disk_size
last_disk_size could be wrong due to concurrently updating, so using
i_sem semaphore to make last_disk_size updating exclusive to fix this
issue.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Thomas Meyer [Sat, 7 Oct 2017 14:02:21 +0000 (16:02 +0200)]
f2fs: Fix bool initialization/comparison
Bool initializations should use true and false. Bool tests don't need
comparisons.
Signed-off-by: Thomas Meyer <thomas@m3y3r.de>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 4 Oct 2017 01:08:37 +0000 (09:08 +0800)]
f2fs: give up CP_TRIMMED_FLAG if it drops discards
In ->umount, once we drop remained discard entries, we should not
set CP_TRIMMED_FLAG with another checkpoint.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 4 Oct 2017 01:08:36 +0000 (09:08 +0800)]
f2fs: trace f2fs_remove_discard
This patch adds tracepoint to trace f2fs_remove_discard.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 4 Oct 2017 01:08:35 +0000 (09:08 +0800)]
f2fs: reduce cmd_lock coverage in __issue_discard_cmd
__submit_discard_cmd may lead long latency due to exhaustion of I/O
request resource in block layer, so issuing all discard under cmd_lock
may lead to hangtask, in order to avoid that, let's reduce it's coverage.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 4 Oct 2017 01:08:34 +0000 (09:08 +0800)]
f2fs: split discard policy
There are many different scenarios such as fstrim, umount, urgent or
background where we will issue discards, actually, they need use
different policy in aspect of io aware, discard granularity, delay
interval and so on. But now they just share one common discard policy,
so there will be race when changing policy in between these scenarios,
the interference of changing discard policy will be very serious.
This patch changes to split discard policy for different scenarios.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 4 Oct 2017 01:08:33 +0000 (09:08 +0800)]
f2fs: wrap discard policy
This patch wraps scattered optional parameters into discard policy as
below, later, with it we expect that we can adjust these parameters with
proper strategy in different scenario.
struct discard_policy {
unsigned int min_interval; /* used for candidates exist */
unsigned int max_interval; /* used for candidates not exist */
unsigned int max_requests; /* # of discards issued per round */
unsigned int io_aware_gran; /* minimum granularity discard not be aware of I/O */
bool io_aware; /* issue discard in idle time */
bool sync; /* submit discard with REQ_SYNC flag */
};
This patch doesn't change any logic of codes.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 4 Oct 2017 01:08:32 +0000 (09:08 +0800)]
f2fs: support issuing/waiting discard in range
Fstrim intends to trim invalid blocks of filesystem only with specified
range and granularity, but actually, it will issue all previous cached
discard commands which may be out-of-range and be with unmatched
granularity, it's unneeded.
In order to fix above issues, this patch introduces new helps to support
to issue and wait discard in range and adds a new fstrim_list for tracking
in-flight discard from ->fstrim.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 29 Sep 2017 05:59:39 +0000 (13:59 +0800)]
f2fs: fix to flush multiple device in checkpoint
If f2fs manages multiple devices, in checkpoint, we need to issue flush
in those devices which contain dirty data/node in their cache before
we write checkpoint region, otherwise, filesystem metadata could be
corrupted if hitting SPO after checkpoint.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 29 Sep 2017 05:59:38 +0000 (13:59 +0800)]
f2fs: enhance multiple device flush
When multiple device feature is enabled, during ->fsync we will issue
flush in all devices to make sure node/data of the file being persisted
into storage. But some flushes of device could be unneeded as file's
data may be not writebacked into those devices. So this patch adds and
manage bitmap per inode in global cache to indicate which device is
dirty and it needs to issue flush during ->fsync, hence, we could improve
performance of fsync in scenario of multiple device.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 29 Sep 2017 05:59:37 +0000 (13:59 +0800)]
f2fs: fix to show ino management cache size correctly
It needs to stat size of ino management cache with all type instead of
orphan ino type.
Fixes:
652be55162dc ("f2fs: show # of orphan inodes")
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 29 Sep 2017 05:59:36 +0000 (13:59 +0800)]
f2fs: drop FI_UPDATE_WRITE tag after f2fs_issue_flush
If we failed to issue flush in ->fsync, we need to keep FI_UPDATE_WRITE
flag to make sure triggering flush in next ->fsync.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 29 Sep 2017 05:59:35 +0000 (13:59 +0800)]
f2fs: obsolete ALLOC_NID_LIST list
As Fan Li reported, there is no user traversing nid_list[ALLOC_NID_LIST]
which is used for tracking preallocated nids. Let's drop it, and only
track preallocated nids in free_nid_root radix-tree.
Reported-by: Fan Li <fanofcode.li@samsung.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Weichao Guo [Fri, 29 Sep 2017 14:43:23 +0000 (22:43 +0800)]
f2fs: convert inline data for direct I/O & FI_NO_PREALLOC
In FI_NO_PREALLOC cases, direct I/O path may allocate blocks for an
inode but keep its inline data flag. This inconsistency may trigger
vfs clear_inode nrpages bug_on when evicting the inode. We should
convert inline data first in this case.
Signed-off-by: Weichao Guo <guoweichao@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Hsiang Kao [Sat, 23 Sep 2017 18:45:42 +0000 (02:45 +0800)]
f2fs: allow readpages with NULL file pointer
Keep in line with the other Linux file system implementations
since page_cache_sync_readahead supports NULL file pointer,
and thus we can readahead data by f2fs itself without file opening
(something like the btrfs behavior).
Signed-off-by: Gao Xiang <gaoxiang25@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Thu, 14 Sep 2017 02:18:01 +0000 (10:18 +0800)]
f2fs: show flush list status in sysfs
This patch adds to show flush list status in sysfs.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 4 Sep 2017 10:58:03 +0000 (18:58 +0800)]
f2fs: introduce read_xattr_block
Commit
ba38c27eb93e ("f2fs: enhance lookup xattr") introduces
lookup_all_xattrs duplicating from read_all_xattrs, which leaves
lots of similar codes in between them, so introduce new help
read_xattr_block to clean up redundant codes.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 4 Sep 2017 10:58:02 +0000 (18:58 +0800)]
f2fs: introduce read_inline_xattr
Commit
ba38c27eb93e ("f2fs: enhance lookup xattr") introduces
lookup_all_xattrs duplicating from read_all_xattrs, which leaves
lots of similar codes in between them, so introduce new help
read_inline_xattr to clean up redundant codes.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 25 Sep 2017 06:17:51 +0000 (14:17 +0800)]
Revert "f2fs: reuse nids more aggressively"
Commit
268344664603 ("f2fs: reuse nids more aggressively") tries to
reuse nids as many as possilbe, in order to mitigate producing obsolete
node pages in page cache.
But acutally, before we reuse the nids and related node page cache,
we will always invalidate that node page, so there will be not any
obsolete node pages in cache.
Let's just revert previous commit, so that nm_i::next_scan_nid can be
increased ascendingly, making __build_free_nids traverses all NAT pages
more easily, finally, free nid bitmap cache can be enabled as soon as
possible.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Sat, 23 Sep 2017 09:02:18 +0000 (17:02 +0800)]
Revert "f2fs: node segment is prior to data segment selected victim"
This reverts commit
b9cd20619e359d199b755543474c3d853c8e3415.
That patch causes much fewer node segments (which can be used for SSR)
than before, and in the corner case (e.g. create and delete *.txt files in
one same directory, there will be very few node segments but many data
segments), if the reserved free segments are all used up during gc, then
the write_checkpoint can still flush dentry pages to data ssr segments,
but will probably fail to flush node pages to node ssr segments, since
there are not enough node ssr segments left (the left ones are all
full).
So revert this patch to give a fair chance to let node segments remain
for SSR, which provides more robustness for corner cases.
Conflicts:
fs/f2fs/gc.c
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Sun, 1 Oct 2017 18:50:16 +0000 (02:50 +0800)]
f2fs: fix potential panic during fstrim
As Ju Hyung Park reported:
"When 'fstrim' is called for manual trim, a BUG() can be triggered
randomly with this patch.
I'm seeing this issue on both x86 Desktop and arm64 Android phone.
On x86 Desktop, this was caused during Ubuntu boot-up. I have a
cronjob installed which calls 'fstrim -v /' during boot. On arm64
Android, this was caused during GC looping with 1ms gc_min_sleep_time
& gc_max_sleep_time."
Root cause of this issue is that f2fs_wait_discard_bios can only be
used by f2fs_put_super, because during put_super there must be no
other referrers, so it can ignore discard entry's reference count
when removing the entry, otherwise in other caller we will hit bug_on
in __remove_discard_cmd as there may be other issuer added reference
count in discard entry.
Thread A Thread B
- issue_discard_thread
- f2fs_ioc_fitrim
- f2fs_trim_fs
- f2fs_wait_discard_bios
- __issue_discard_cmd
- __submit_discard_cmd
- __wait_discard_cmd
- dc->ref++
- __wait_one_discard_bio
- __wait_discard_cmd
- __remove_discard_cmd
- f2fs_bug_on(sbi, dc->ref)
Fixes:
969d1b180d987c2be02de890d0fff0f66a0e80de
Reported-by: Ju Hyung Park <qkrwngud825@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 12 Sep 2017 13:35:12 +0000 (21:35 +0800)]
f2fs: hurry up to issue discard after io interruption
Once we encounter I/O interruption during issuing discards, we will delay
long time before next round, but if system status is I/O idle during the
time, it may loses opportunity to issue discards. So this patch changes
to hurry up to issue discard after io interruption.
Besides, this patch also fixes to issue discards accurately with assigned
rate.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 12 Sep 2017 06:25:35 +0000 (14:25 +0800)]
f2fs: fix to show correct discard_granularity in sysfs
Fix below incorrect display when reading discard_granularity sysfs node.
$ cat /sys/fs/f2fs/<device>/discard_granularity
$ 16
$ echo 32 > /sys/fs/f2fs/<device>/discard_granularity
$ cat /sys/fs/f2fs/<device>/discard_granularity
$ 16
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 12 Sep 2017 06:04:05 +0000 (14:04 +0800)]
f2fs: detect dirty inode in evict_inode
Add a bugon in f2fs_evict_inode to detect inconsistent status between
inode cache and related node page cache.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Mon, 11 Sep 2017 07:30:28 +0000 (16:30 +0900)]
f2fs: clear radix tree dirty tag of pages whose dirty flag is cleared
On a senario like writing out the first dirty page of the inode
as the inline data, we only cleared dirty flags of the pages, but
didn't clear the dirty tags of those pages in the radix tree.
If we don't clear the dirty tags of the pages in the radix tree, the
inodes which contain the pages will be marked with I_DIRTY_PAGES again
and again, and writepages() for the inodes will be invoked in every
writeback period. As a result, nothing will be done in every
writepages() for the inodes and it will just consume CPU time
meaninglessly.
Signed-off-by: Daeho Jeong <daeho.jeong@samsung.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 9 Sep 2017 18:11:04 +0000 (11:11 -0700)]
f2fs: speed up gc_urgent mode with SSR
This patch activates SSR in gc_urgent mode.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 9 Sep 2017 19:03:23 +0000 (12:03 -0700)]
f2fs: better to wait for fstrim completion
In android, we'd better wait for fstrim completion instead of issuing the
discard commands asynchronous.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlei He [Thu, 7 Sep 2017 02:40:54 +0000 (10:40 +0800)]
f2fs: avoid race in between read xattr & write xattr
Thread A: Thread B:
-f2fs_getxattr
-lookup_all_xattrs
-xnid = F2FS_I(inode)->i_xattr_nid;
-f2fs_setxattr
-__f2fs_setxattr
-write_all_xattrs
-truncate_xattr_node
... ...
-write_checkpoint
... ...
-alloc_nid <- nid reuse
-get_node_page
-f2fs_bug_on <- nid != node_footer->nid
It's need a rw_sem to avoid the race
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 7 Sep 2017 04:04:44 +0000 (21:04 -0700)]
f2fs: make get_lock_data_page to handle encrypted inode
This patch refactors get_lock_data_page() to handle encryption case directly.
In order to do that, it introduces common f2fs_submit_page_read().
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 6 Sep 2017 00:04:35 +0000 (17:04 -0700)]
f2fs: use generic terms used for encrypted block management
This patch renames functions regarding to buffer management via META_MAPPING
used for encrypted blocks especially. We can actually use them in generic way.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 5 Sep 2017 23:54:24 +0000 (16:54 -0700)]
f2fs: introduce f2fs_encrypted_file for clean-up
This patch replaces (f2fs_encrypted_inode() && S_ISREG()) with
f2fs_encrypted_file(), which gives no functional change.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Mon, 4 Sep 2017 03:10:18 +0000 (11:10 +0800)]
Revert "f2fs: add a new function get_ssr_cost"
This reverts commit
b7b7c4cf1c9ef0272a65f1480457cbfdadcda19d.
se->ckpt_valid_blocks will never be smaller than se->valid_blocks, so just
remove get_ssr_cost.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Arvind Yadav [Thu, 31 Aug 2017 09:36:24 +0000 (15:06 +0530)]
f2fs: constify super_operations
super_operations are not supposed to change at runtime.
"struct super_block" working with super_operations provided
by <linux/fs.h> work with const super_operations. So mark
the non-const structs as const
Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Thu, 31 Aug 2017 10:56:06 +0000 (18:56 +0800)]
f2fs: fix to wake up all sleeping flusher
In scenario of remount_ro vs flush, after flush_thread exits in
->remount_fs, flusher will only clean up golbal issue_list, but
without waking up flushers waiting on that list, result in hang
related user threads.
In order to fix this issue, this patch enables the flusher to
take charge of issue_flush thread: executes merged flush command,
and wake up all sleeping flushers.
Fixes:
5eba8c5d1fb3 ("f2fs: fix to access nullified flush_cmd_control pointer")
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Thu, 31 Aug 2017 10:56:05 +0000 (18:56 +0800)]
f2fs: avoid race in between atomic_read & atomic_inc
Previously, we will miss merging flush command during fsync due to below
race condition:
Thread A Thread B Thread C
- f2fs_issue_flush
- atomic_read(&issing_flush)
- f2fs_issue_flush
- atomic_read(&issing_flush)
- f2fs_issue_flush
- atomic_read(&issing_flush)
- atomic_inc(&issing_flush)
- atomic_inc(&issing_flush)
- atomic_inc(&issing_flush)
- submit_flush_wait
- submit_flush_wait
- submit_flush_wait
It needs to use atomic_inc_return instead to avoid such race.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 30 Aug 2017 10:04:48 +0000 (18:04 +0800)]
f2fs: remove unneeded parameter of change_curseg
allocate_segment_by_default is the only caller of change_curseg passing
@reuse with 'false', but commit
763bfe1bc575 ("f2fs: remove reusing any
prefree segments") removes the calling, after that, @reuse in
change_curseg always be true, so, let's clean up the unneeded parameter.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 30 Aug 2017 10:04:47 +0000 (18:04 +0800)]
f2fs: update i_flags correctly
f2fs enables hash-indexed directory by default, so we need to tag
FS_INDEX_FL in inode::i_flags during directory creataion, in order
to show correct status of inode in lsattr:
Before:
------------------- /mnt/f2fs/dir/
After:
-----------I------- /mnt/f2fs/dir/
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 31 Aug 2017 23:54:51 +0000 (16:54 -0700)]
f2fs: don't check inode's checksum if it was dirtied or writebacked
If another thread already made the page dirtied or writebacked, we must avoid
to verify checksum. If we got an error, we need to remove its uptodate as well.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 31 Aug 2017 18:10:58 +0000 (11:10 -0700)]
f2fs: don't need to update inode checksum for recovery
This patch fixes "f2fs: support inode checksum".
The recovered inode page will be rewritten with valid checksum.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 23 Aug 2017 10:23:25 +0000 (18:23 +0800)]
f2fs: trigger fdatasync for non-atomic_write file
Sqlite only cares about synchronization of file data instead of other data
unrelated attribute of inode, so in commit flow, call fdatasync is enough.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 23 Aug 2017 10:23:24 +0000 (18:23 +0800)]
f2fs: fix to avoid race in between aio and gc
We won't wait DIO synchronously when doing AIO, so there will be potential
IO reorder in between AIO and GC, which will cause data corruption.
This patch adds inode_dio_wait to serialize aio and data GC to avoid this
issue.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 23 Aug 2017 04:15:43 +0000 (21:15 -0700)]
f2fs: wake up discard_thread iff there is a candidate
This patch fixes to avoid needless wake ups.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 21 Aug 2017 20:51:32 +0000 (13:51 -0700)]
f2fs: return error when accessing insane flie offset
If file offset is insane, we have to return error instead of kernel panic.
Reported-by: Eric Zhang <followme999@163.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 18 Aug 2017 08:20:33 +0000 (16:20 +0800)]
f2fs: trigger normal fsync for non-atomic_write file
If file was not opened with atomic write mode, but user uses atomic write
ioctl to fsync datas, in the flow, we should not fsync that file with
atomic write mode.
Fixes:
608514deba38 ("f2fs: set fsync mark only for the last dnode")
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 18 Aug 2017 15:37:36 +0000 (23:37 +0800)]
f2fs: clear FI_HOT_DATA correctly
This patch fixes to clear FI_HOT_DATA correctly in below path:
- error handling in f2fs_ioc_start_atomic_write
- after commit atomic write in f2fs_ioc_commit_atomic_write
- after drop atomic write in drop_inmem_pages
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 21 Aug 2017 14:53:45 +0000 (22:53 +0800)]
f2fs: fix out-of-order execution in f2fs_issue_flush
In f2fs_issue_flush, due to out-of-order execution of CPU, wake_up can
be called before we insert issue_list, result in long latency of
wait_for_completion. Fix this by adding smp_mb() to force the order of
related codes.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 16 Aug 2017 04:27:19 +0000 (21:27 -0700)]
f2fs: issue discard commands if gc_urgent is set
It's time to issue all the discard commands, if user sets the idle time.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 7 Aug 2017 15:09:56 +0000 (23:09 +0800)]
f2fs: introduce discard_granularity sysfs entry
Commit
d618ebaf0aa8 ("f2fs: enable small discard by default") enables
f2fs to issue 4K size discard in real-time discard mode. However, issuing
smaller discard may cost more lifetime but releasing less free space in
flash device. Since f2fs has ability of separating hot/cold data and
garbage collection, we can expect that small-sized invalid region would
expand soon with OPU, deletion or garbage collection on valid datas, so
it's better to delay or skip issuing smaller size discards, it could help
to reduce overmuch consumption of IO bandwidth and lifetime of flash
storage.
This patch makes f2fs selectng 64K size as its default minimal
granularity, and issue discard with the size which is not smaller than
minimal granularity. Also it exposes discard granularity as sysfs entry
for configuration in different scenario.
Jaegeuk Kim:
We must issue all the accumulated discard commands when fstrim is called.
So, I've added pend_list_tag[] to indicate whether we should issue the
commands or not. If tag sets P_ACTIVE or P_TRIM, we have to issue them.
P_TRIM is set once at a time, given fstrim trigger.
In addition, issue_discard_thread is calling too much due to the number of
discard commands remaining in the pending list. I added a timer to control
it likewise gc_thread.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Mon, 14 Aug 2017 08:52:43 +0000 (16:52 +0800)]
f2fs: remove unused function overprovision_sections
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sun, 13 Aug 2017 04:33:23 +0000 (21:33 -0700)]
f2fs: check hot_data for roll-forward recovery
We need to check HOT_DATA to truncate any previous data block when doing
roll-forward recovery.
Cc: <stable@vger.kernel.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 11 Aug 2017 10:00:15 +0000 (18:00 +0800)]
f2fs: add tracepoint for f2fs_gc
This patch adds tracepoint for f2fs_gc.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 8 Aug 2017 11:09:08 +0000 (19:09 +0800)]
f2fs: retry to revoke atomic commit in -ENOMEM case
During atomic committing, if we encounter -ENOMEM in revoke path, it's
better to give a chance to retry revoking.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 11 Aug 2017 00:35:04 +0000 (17:35 -0700)]
f2fs: let fill_super handle roll-forward errors
If we set CP_ERROR_FLAG in roll-forward error, f2fs is no longer to proceed
any IOs due to f2fs_cp_error(). But, for example, if some stale data is involved
on roll-forward process, we're able to get -ENOENT, getting fs stuck.
If we get any error, let fill_super set SBI_NEED_FSCK and try to recover back
to stable point.
Cc: <stable@vger.kernel.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Qiuyang Sun [Wed, 9 Aug 2017 09:27:30 +0000 (17:27 +0800)]
f2fs: merge equivalent flags F2FS_GET_BLOCK_[READ|DIO]
Currently, the two flags F2FS_GET_BLOCK_[READ|DIO] are totally equivalent
and can be used interchangably in all scenarios they are involved in.
Neither of the flags is referenced in f2fs_map_blocks(), making them both
the default case. To remove the ambiguity, this patch merges both flags
into F2FS_GET_BLOCK_DEFAULT, and introduces an enum for all distinct flags.
Signed-off-by: Qiuyang Sun <sunqiuyang@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 8 Aug 2017 02:54:31 +0000 (10:54 +0800)]
f2fs: support journalled quota
This patch supports to enable f2fs to accept quota information through
mount option:
- {usr,grp,prj}jquota=<quota file path>
- jqfmt=<quota type>
Then, in ->mount flow, we can recover quota file during log replaying,
by this, journelled quota can be supported.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: Fix wrong return values.]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 7 Aug 2017 15:12:46 +0000 (23:12 +0800)]
f2fs: fix potential overflow when adjusting GC cycle
While comparing signed and unsigned variables, compiler will converts the
signed value to unsigned one, due to this reason, {in,de}crease_sleep_time
may return overflowed result.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 7 Aug 2017 08:37:59 +0000 (16:37 +0800)]
f2fs: avoid unneeded sync on quota file
We only need to sync quota file with appointed quota type instead of all
types in f2fs_quota_{on,off}.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 7 Aug 2017 05:09:00 +0000 (22:09 -0700)]
f2fs: introduce gc_urgent mode for background GC
This patch adds a sysfs entry to control urgent mode for background GC.
If this is set, background GC thread conducts GC with gc_urgent_sleep_time
all the time.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 5 Aug 2017 21:25:08 +0000 (14:25 -0700)]
f2fs: use IPU for cold files
We expect cold files write data sequentially, but sometimes some of small data
can be updated, which incurs fragmentation.
Let's avoid that.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Fri, 4 Aug 2017 09:07:15 +0000 (17:07 +0800)]
f2fs: fix the size value in __check_sit_bitmap
The current size value is not correct and will miss bitmap check.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 2 Aug 2017 15:21:48 +0000 (23:21 +0800)]
f2fs: add app/fs io stat
This patch enables inner app/fs io stats and introduces below virtual fs
nodes for exposing stats info:
/sys/fs/f2fs/<dev>/iostat_enable
/proc/fs/f2fs/<dev>/iostat_info
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix wrong stat assignment]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Wed, 2 Aug 2017 14:16:54 +0000 (22:16 +0800)]
f2fs: do not change the valid_block value if cur_valid_map was wrongly set or cleared
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Wed, 2 Aug 2017 13:20:13 +0000 (21:20 +0800)]
f2fs: update cur_valid_map_mir together with cur_valid_map
When cur_valid_map passes the f2fs_test_and_set(,clear)_bit test,
cur_valid_map_mir update is skipped unlikely, so fix it. The fix
now changes the mirror check together with cur_valid_map all the
time.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: Fix unused variable and add unlikely for corner condition.]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 3 Aug 2017 03:58:29 +0000 (20:58 -0700)]
f2fs: use printk_ratelimited for f2fs_msg
This patch reduces contention of printks.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 22 Jul 2017 00:14:09 +0000 (17:14 -0700)]
f2fs: expose features to sysfs entry
This patch exposes what features are supported by current f2fs build to sysfs
entry via:
/sys/fs/f2fs/features/
/sys/fs/f2fs/dev/features
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 31 Jul 2017 12:19:09 +0000 (20:19 +0800)]
f2fs: support inode checksum
This patch adds to support inode checksum in f2fs.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix verification flow]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sun, 30 Jul 2017 16:45:14 +0000 (09:45 -0700)]
f2fs: return wrong error number on f2fs_quota_write
This must return size, not error number.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlong Song [Thu, 27 Jul 2017 12:11:00 +0000 (20:11 +0800)]
f2fs: provide f2fs_balance_fs to __write_node_page
Let node writeback also do f2fs_balance_fs to ensure there are always enough free
segments.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 28 Jul 2017 16:32:53 +0000 (00:32 +0800)]
f2fs: introduce f2fs_statfs_project
This patch introduces f2fs_statfs_project, it enables to show usage
status of directory tree which is limited with project quota.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 28 Jul 2017 09:29:12 +0000 (02:29 -0700)]
f2fs: don't need to wait for node writes for atomic write
We have a node chain to serialize node block writes, so if any IOs for
node block writes are reordered, we'll get broken node chain. IOWs,
roll-forward recovery will see all or none node blocks given fsync
mark.
E.g.,
Node chain consists of:
N1 -> N2 -> N3 -> NFSYNC -> N1' -> N2' -> N'FSYNC
Reordered to:
1) N1 -> N2 -> N3 -> N2' -> NFSYNC -> N'FSYNC -> power-cut
2) N1 -> N2 -> N3 -> N1' -> NFSYNC -> power-cut
3) N1 -> N2 -> NFSYNC -> N1' -> N'FSYNC -> N3 -> power-cut
4) N1 -> NFSYNC -> N1' -> N2' -> N'FSYNC -> N3 -> power-cut
Roll-forward recovery can proceed to:
1) N1 -> N2 -> N3 -> NFSYNC -> X
2) N1 -> N2 -> N3 -> NFSYNC -> N1' -> X
3) N1 -> N2 -> N3 -> FSYNC -> N1' -> X
4) N1 -> X
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 26 Jul 2017 18:24:13 +0000 (11:24 -0700)]
f2fs: avoid naming confusion of sysfs init
This patch changes the function names of sysfs init to follow ext4.
f2fs_init_sysfs <-> f2fs_register_sysfs
f2fs_exit_sysfs <-> f2fs_unregister_sysfs
Suggested-by: Chao Yu <yuchao0@huawei.com>
Reivewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 25 Jul 2017 16:01:41 +0000 (00:01 +0800)]
f2fs: support project quota
This patch adds to support plain project quota.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 24 Jul 2017 09:12:06 +0000 (17:12 +0800)]
f2fs: record quota during dot{,dot} recovery
In ->lookup(), we will have a try to recover dot or dotdot for
corrupted directory, once disk quota is on, if it allocates new
block during dotdot recovery, we need to record disk quota info
for the allocation, so this patch fixes this issue by adding
missing dquot_initialize() in __recover_dot_dentries.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 18 Jul 2017 16:19:06 +0000 (00:19 +0800)]
f2fs: enhance on-disk inode structure scalability
This patch add new flag F2FS_EXTRA_ATTR storing in inode.i_inline
to indicate that on-disk structure of current inode is extended.
In order to extend, we changed the inode structure a bit:
Original one:
struct f2fs_inode {
...
struct f2fs_extent i_ext;
__le32 i_addr[DEF_ADDRS_PER_INODE];
__le32 i_nid[DEF_NIDS_PER_INODE];
}
Extended one:
struct f2fs_inode {
...
struct f2fs_extent i_ext;
union {
struct {
__le16 i_extra_isize;
__le16 i_padding;
__le32 i_extra_end[0];
};
__le32 i_addr[DEF_ADDRS_PER_INODE];
};
__le32 i_nid[DEF_NIDS_PER_INODE];
}
Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
we can calculate actual size of reserved space in i_addr, available
attribute fields included in total extra attribute fields for current
inode can be described as below:
+--------------------+
| .i_mode |
| ... |
| .i_ext |
+--------------------+
| .i_extra_isize |-----+
| .i_padding | |
| .i_prjid | |
| .i_atime_extra | |
| .i_ctime_extra | |
| .i_mtime_extra |<----+
| .i_inode_cs |<----- store blkaddr/inline from here
| .i_xattr_cs |
| ... |
+--------------------+
| |
| block address |
| |
+--------------------+
| .i_nid |
+--------------------+
| node_footer |
| (nid, ino, offset) |
+--------------------+
Hence, with this patch, we would enhance scalability of f2fs inode for
storing more newly added attribute.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 18 Jul 2017 16:19:05 +0000 (00:19 +0800)]
f2fs: make max inline size changeable
This patch tries to make below macros calculating max inline size,
inline dentry field size considerring reserving size-changeable
space:
- MAX_INLINE_DATA
- NR_INLINE_DENTRY
- INLINE_DENTRY_BITMAP_SIZE
- INLINE_RESERVED_SIZE
Then, when inline_{data,dentry} options is enabled, it allows us to
reserve inline space with different size flexibly for adding newly
introduced inode attribute.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 21 Jul 2017 19:58:59 +0000 (12:58 -0700)]
f2fs: add ioctl to expose current features
This patch adds an ioctl to provide feature information to user.
For exapmle, SQLite can use this ioctl to detect whether f2fs support atomic
write or not.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Sat, 22 Jul 2017 00:52:23 +0000 (08:52 +0800)]
f2fs: make background threads of f2fs being aware of freezing
When ->freeze_fs is called from lvm for doing snapshot, it needs to
make sure there will be no more changes in filesystem's data, however,
previously, background threads like GC thread wasn't aware of freezing,
so in environment with active background threads, data of snapshot
becomes unstable.
This patch fixes this issue by adding sb_{start,end}_intwrite in
below background threads:
- GC thread
- flush thread
- discard thread
Note that, don't use sb_start_intwrite() in gc_thread_func() due to:
generic/241 reports below bug:
======================================================
WARNING: possible circular locking dependency detected
4.13.0-rc1+ #32 Tainted: G O
------------------------------------------------------
f2fs_gc-250:0/22186 is trying to acquire lock:
(&sbi->gc_mutex){+.+...}, at: [<
f8fa7f0b>] f2fs_sync_fs+0x7b/0x1b0 [f2fs]
but task is already holding lock:
(sb_internal#2){++++.-}, at: [<
f8fb5609>] gc_thread_func+0x159/0x4a0 [f2fs]
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (sb_internal#2){++++.-}:
__lock_acquire+0x405/0x7b0
lock_acquire+0xae/0x220
__sb_start_write+0x11d/0x1f0
f2fs_evict_inode+0x2d6/0x4e0 [f2fs]
evict+0xa8/0x170
iput+0x1fb/0x2c0
f2fs_sync_inode_meta+0x3f/0xf0 [f2fs]
write_checkpoint+0x1b1/0x750 [f2fs]
f2fs_sync_fs+0x85/0x1b0 [f2fs]
f2fs_do_sync_file.isra.24+0x137/0xa30 [f2fs]
f2fs_sync_file+0x34/0x40 [f2fs]
vfs_fsync_range+0x4a/0xa0
do_fsync+0x3c/0x60
SyS_fdatasync+0x15/0x20
do_fast_syscall_32+0xa1/0x1b0
entry_SYSENTER_32+0x4c/0x7b
-> #1 (&sbi->cp_mutex){+.+...}:
__lock_acquire+0x405/0x7b0
lock_acquire+0xae/0x220
__mutex_lock+0x4f/0x830
mutex_lock_nested+0x25/0x30
write_checkpoint+0x2f/0x750 [f2fs]
f2fs_sync_fs+0x85/0x1b0 [f2fs]
sync_filesystem+0x67/0x80
generic_shutdown_super+0x27/0x100
kill_block_super+0x22/0x50
kill_f2fs_super+0x3a/0x40 [f2fs]
deactivate_locked_super+0x3d/0x70
deactivate_super+0x40/0x60
cleanup_mnt+0x39/0x70
__cleanup_mnt+0x10/0x20
task_work_run+0x69/0x80
exit_to_usermode_loop+0x57/0x92
do_fast_syscall_32+0x18c/0x1b0
entry_SYSENTER_32+0x4c/0x7b
-> #0 (&sbi->gc_mutex){+.+...}:
validate_chain.isra.36+0xc50/0xdb0
__lock_acquire+0x405/0x7b0
lock_acquire+0xae/0x220
__mutex_lock+0x4f/0x830
mutex_lock_nested+0x25/0x30
f2fs_sync_fs+0x7b/0x1b0 [f2fs]
f2fs_balance_fs_bg+0xb9/0x200 [f2fs]
gc_thread_func+0x302/0x4a0 [f2fs]
kthread+0xe9/0x120
ret_from_fork+0x19/0x24
other info that might help us debug this:
Chain exists of:
&sbi->gc_mutex --> &sbi->cp_mutex --> sb_internal#2
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(sb_internal#2);
lock(&sbi->cp_mutex);
lock(sb_internal#2);
lock(&sbi->gc_mutex);
*** DEADLOCK ***
1 lock held by f2fs_gc-250:0/22186:
#0: (sb_internal#2){++++.-}, at: [<
f8fb5609>] gc_thread_func+0x159/0x4a0 [f2fs]
stack backtrace:
CPU: 2 PID: 22186 Comm: f2fs_gc-250:0 Tainted: G O 4.13.0-rc1+ #32
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Call Trace:
dump_stack+0x5f/0x92
print_circular_bug+0x1b3/0x1bd
validate_chain.isra.36+0xc50/0xdb0
? __this_cpu_preempt_check+0xf/0x20
__lock_acquire+0x405/0x7b0
lock_acquire+0xae/0x220
? f2fs_sync_fs+0x7b/0x1b0 [f2fs]
__mutex_lock+0x4f/0x830
? f2fs_sync_fs+0x7b/0x1b0 [f2fs]
mutex_lock_nested+0x25/0x30
? f2fs_sync_fs+0x7b/0x1b0 [f2fs]
f2fs_sync_fs+0x7b/0x1b0 [f2fs]
f2fs_balance_fs_bg+0xb9/0x200 [f2fs]
gc_thread_func+0x302/0x4a0 [f2fs]
? preempt_schedule_common+0x2f/0x4d
? f2fs_gc+0x540/0x540 [f2fs]
kthread+0xe9/0x120
? f2fs_gc+0x540/0x540 [f2fs]
? kthread_create_on_node+0x30/0x30
ret_from_fork+0x19/0x24
The deadlock occurs in below condition:
GC Thread Thread B
- sb_start_intwrite
- f2fs_sync_file
- f2fs_sync_fs
- mutex_lock(&sbi->gc_mutex)
- write_checkpoint
- block_operations
- f2fs_sync_inode_meta
- iput
- sb_start_intwrite
- mutex_lock(&sbi->gc_mutex)
Fix this by altering sb_start_intwrite to sb_start_write_trylock.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 25 Jul 2017 02:46:29 +0000 (19:46 -0700)]
f2fs: don't give partially written atomic data from process crash
This patch resolves the below scenario.
== Process 1 == == Process 2 ==
open(w) open(rw)
begin
write(new_#1)
process_crash
f_op->flush
locks_remove_posix
f_op>release
read (new_#1)
In order to avoid corrupted database caused by new_#1, we must do roll-back
at process_crash time. In order to check that, this patch keeps task which
triggers transaction begin, and does roll-back in f_op->flush before removing
file locks.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 19 Jul 2017 17:59:55 +0000 (10:59 -0700)]
f2fs: give a try to do atomic write in -ENOMEM case
It'd be better to retry writing atomic pages when we get -ENOMEM.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Ernesto A. Fernández [Mon, 24 Jul 2017 01:32:54 +0000 (22:32 -0300)]
f2fs: preserve i_mode if __f2fs_set_acl() fails
When changing a file's acl mask, __f2fs_set_acl() will first set the
group bits of i_mode to the value of the mask, and only then set the
actual extended attribute representing the new acl.
If the second part fails (due to lack of space, for example) and the
file had no acl attribute to begin with, the system will from now on
assume that the mask permission bits are actual group permission bits,
potentially granting access to the wrong users.
Prevent this by only changing the inode mode after the acl has been set.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlei He [Tue, 18 Jul 2017 01:48:12 +0000 (09:48 +0800)]
f2fs: alloc new nids for xattr block in recovery
recovery file A: recovery file B:
-get_dnode_of_data
-alloc_nid
-recover_xattr_data
-set_node_addr(sbi, &ni, NEW_ADDR, false);
--->bug_on for nid has been used by file A
In recovery process, new allocated node blocks may "reuse" xattr block
nids, this patch alloc new nids for xattr blocks in recovery process to
avoid this problem.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Sun, 16 Jul 2017 07:08:54 +0000 (15:08 +0800)]
f2fs: spread struct f2fs_dentry_ptr for inline path
Use f2fs_dentry_ptr structure to indicate inline dentry structure as
much as possible, so we can wrap inline dentry with size-fixed fields
to the one with size-changeable fields. With this change, we can
handle size-changeable inline dentry more easily.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Yunlei He [Mon, 17 Jul 2017 11:16:11 +0000 (19:16 +0800)]
f2fs: remove unused input parameter
This patch remove unused input parameter in function
new_node_page.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Yong Sheng <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 14 Jul 2017 18:45:21 +0000 (11:45 -0700)]
f2fs: avoid cpu lockup
Before retrying to flush data or dentry pages, we need to release cpu in order
to prevent watchdog.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 14 Jul 2017 00:45:21 +0000 (17:45 -0700)]
f2fs: include seq_file.h for sysfs.c
This patch includes seq_file.h to avoid compile error.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 11 Jul 2017 21:56:49 +0000 (14:56 -0700)]
f2fs: Don't clear SGID when inheriting ACLs
This patch copies commit
b7f8a09f80:
"btrfs: Don't clear SGID when inheriting ACLs" written by Jan.
Fixes:
073931017b49d9458aa351605b43a7e34598caef
CC: stable@vger.kernel.org
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Luis Henriques [Tue, 11 Jul 2017 16:30:33 +0000 (17:30 +0100)]
f2fs: remove extra inode_unlock() in error path
This commit removes an extra inode_unlock() that is being done in function
f2fs_ioc_setflags error path. While there, get rid of a useless 'out'
label as well.
Fixes:
0abd675e97e6 ("f2fs: support plain user/group quota")
Signed-off-by: Luis Henriques <lhenriques@suse.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daniel Walter [Mon, 19 Jun 2017 07:27:58 +0000 (09:27 +0200)]
fscrypt: add support for AES-128-CBC
fscrypt provides facilities to use different encryption algorithms which
are selectable by userspace when setting the encryption policy. Currently,
only AES-256-XTS for file contents and AES-256-CBC-CTS for file names are
implemented. This is a clear case of kernel offers the mechanism and
userspace selects a policy. Similar to what dm-crypt and ecryptfs have.
This patch adds support for using AES-128-CBC for file contents and
AES-128-CBC-CTS for file name encryption. To mitigate watermarking
attacks, IVs are generated using the ESSIV algorithm. While AES-CBC is
actually slightly less secure than AES-XTS from a security point of view,
there is more widespread hardware support. Using AES-CBC gives us the
acceptable performance while still providing a moderate level of security
for persistent storage.
Especially low-powered embedded devices with crypto accelerators such as
CAAM or CESA often only support AES-CBC. Since using AES-CBC over AES-XTS
is basically thought of a last resort, we use AES-128-CBC over AES-256-CBC
since it has less encryption rounds and yields noticeable better
performance starting from a file size of just a few kB.
Signed-off-by: Daniel Walter <dwalter@sigma-star.at>
[david@sigma-star.at: addressed review comments]
Signed-off-by: David Gstir <david@sigma-star.at>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Tue, 23 May 2017 01:14:06 +0000 (18:14 -0700)]
fscrypt: inline fscrypt_free_filename()
fscrypt_free_filename() only needs to do a kfree() of crypto_buf.name,
which works well as an inline function. We can skip setting the various
pointers to NULL, since no user cares about it (the name is always freed
just before it goes out of scope).
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: David Gstir <david@sigma-star.at>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jaegeuk Kim [Tue, 11 Jul 2017 02:16:28 +0000 (19:16 -0700)]
f2fs: make more close to v4.13-rc1
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Sat, 8 Jul 2017 16:13:07 +0000 (00:13 +0800)]
f2fs: support plain user/group quota
This patch adds to support plain user/group quota.
Change Note by Jaegeuk Kim.
- Use f2fs page cache for quota files in order to consider garbage collection.
so, quota files are not tolerable for sudden power-cuts, so user needs to do
quotacheck.
- setattr() calls dquot_transfer which will transfer inode->i_blocks.
We can't reclaim that during f2fs_evict_inode(). So, we need to count
node blocks as well in order to match i_blocks with dquot's space.
Note that, Chao wrote a patch to count inode->i_blocks without inode block.
(f2fs: don't count inode block in in-memory inode.i_blocks)
- in f2fs_remount, we need to make RW in prior to dquot_resume.
- handle fault_injection case during f2fs_quota_off_umount
- TODO: Project quota
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 22 Jun 2017 00:52:39 +0000 (17:52 -0700)]
f2fs: avoid deadlock caused by lock order of page and lock_op
- punch_hole
- fill_zero
- f2fs_lock_op
- get_new_data_page
- lock_page
- f2fs_write_data_pages
- lock_page
- do_write_data_page
- f2fs_lock_op
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 7 Jul 2017 06:10:15 +0000 (14:10 +0800)]
f2fs: use spin_{,un}lock_irq{save,restore}
generic/361 reports below warning, this is because: once, there is
someone entering into critical region of sbi.cp_lock, if write_end_io.
f2fs_stop_checkpoint is invoked from an triggered IRQ, we will encounter
deadlock.
So this patch changes to use spin_{,un}lock_irq{save,restore} to create
critical region without IRQ enabled to avoid potential deadlock.
irq event stamp:
83391573
loop: Write error at byte offset
438729728, length 1024.
hardirqs last enabled at (
83391573): [<
c1809752>] restore_all+0xf/0x65
hardirqs last disabled at (
83391572): [<
c1809eac>] reschedule_interrupt+0x30/0x3c
loop: Write error at byte offset
438860288, length 1536.
softirqs last enabled at (
83389244): [<
c180cc4e>] __do_softirq+0x1ae/0x476
softirqs last disabled at (
83389237): [<
c101ca7c>] do_softirq_own_stack+0x2c/0x40
loop: Write error at byte offset
438990848, length 2048.
================================
WARNING: inconsistent lock state
4.12.0-rc2+ #30 Tainted: G O
--------------------------------
inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage.
xfs_io/7959 [HC1[1]:SC0[0]:HE0:SE1] takes:
(&(&sbi->cp_lock)->rlock){?.+...}, at: [<
f96f96cc>] f2fs_stop_checkpoint+0x1c/0x50 [f2fs]
{HARDIRQ-ON-W} state was registered at:
__lock_acquire+0x527/0x7b0
lock_acquire+0xae/0x220
_raw_spin_lock+0x42/0x50
do_checkpoint+0x165/0x9e0 [f2fs]
write_checkpoint+0x33f/0x740 [f2fs]
__f2fs_sync_fs+0x92/0x1f0 [f2fs]
f2fs_sync_fs+0x12/0x20 [f2fs]
sync_filesystem+0x67/0x80
generic_shutdown_super+0x27/0x100
kill_block_super+0x22/0x50
kill_f2fs_super+0x3a/0x40 [f2fs]
deactivate_locked_super+0x3d/0x70
deactivate_super+0x40/0x60
cleanup_mnt+0x39/0x70
__cleanup_mnt+0x10/0x20
task_work_run+0x69/0x80
exit_to_usermode_loop+0x57/0x85
do_fast_syscall_32+0x18c/0x1b0
entry_SYSENTER_32+0x4c/0x7b
irq event stamp:
1957420
hardirqs last enabled at (
1957419): [<
c1808f37>] _raw_spin_unlock_irq+0x27/0x50
hardirqs last disabled at (
1957420): [<
c1809f9c>] call_function_single_interrupt+0x30/0x3c
softirqs last enabled at (
1953784): [<
c180cc4e>] __do_softirq+0x1ae/0x476
softirqs last disabled at (
1953773): [<
c101ca7c>] do_softirq_own_stack+0x2c/0x40
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(&(&sbi->cp_lock)->rlock);
<Interrupt>
lock(&(&sbi->cp_lock)->rlock);
*** DEADLOCK ***
2 locks held by xfs_io/7959:
#0: (sb_writers#13){.+.+.+}, at: [<
c11fd7ca>] vfs_write+0x16a/0x190
#1: (&sb->s_type->i_mutex_key#16){+.+.+.}, at: [<
f96e33f5>] f2fs_file_write_iter+0x25/0x140 [f2fs]
stack backtrace:
CPU: 2 PID: 7959 Comm: xfs_io Tainted: G O 4.12.0-rc2+ #30
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Call Trace:
dump_stack+0x5f/0x92
print_usage_bug+0x1d3/0x1dd
? check_usage_backwards+0xe0/0xe0
mark_lock+0x23d/0x280
__lock_acquire+0x699/0x7b0
? __this_cpu_preempt_check+0xf/0x20
? trace_hardirqs_off_caller+0x91/0xe0
lock_acquire+0xae/0x220
? f2fs_stop_checkpoint+0x1c/0x50 [f2fs]
_raw_spin_lock+0x42/0x50
? f2fs_stop_checkpoint+0x1c/0x50 [f2fs]
f2fs_stop_checkpoint+0x1c/0x50 [f2fs]
f2fs_write_end_io+0x147/0x150 [f2fs]
bio_endio+0x7a/0x1e0
blk_update_request+0xad/0x410
blk_mq_end_request+0x16/0x60
lo_complete_rq+0x3c/0x70
__blk_mq_complete_request_remote+0x11/0x20
flush_smp_call_function_queue+0x6d/0x120
? debug_smp_processor_id+0x12/0x20
generic_smp_call_function_single_interrupt+0x12/0x30
smp_call_function_single_interrupt+0x25/0x40
call_function_single_interrupt+0x37/0x3c
EIP: _raw_spin_unlock_irq+0x2d/0x50
EFLAGS:
00000296 CPU: 2
EAX:
00000001 EBX:
d2ccc51c ECX:
00000001 EDX:
c1aacebd
ESI:
00000000 EDI:
00000000 EBP:
c96c9d1c ESP:
c96c9d18
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
? inherit_task_group.isra.98.part.99+0x6b/0xb0
__add_to_page_cache_locked+0x1d4/0x290
add_to_page_cache_lru+0x38/0xb0
pagecache_get_page+0x8e/0x200
f2fs_write_begin+0x96/0xf00 [f2fs]
? trace_hardirqs_on_caller+0xdd/0x1c0
? current_time+0x17/0x50
? trace_hardirqs_on+0xb/0x10
generic_perform_write+0xa9/0x170
__generic_file_write_iter+0x1a2/0x1f0
? f2fs_preallocate_blocks+0x137/0x160 [f2fs]
f2fs_file_write_iter+0x6e/0x140 [f2fs]
? __lock_acquire+0x429/0x7b0
__vfs_write+0xc1/0x140
vfs_write+0x9b/0x190
SyS_pwrite64+0x63/0xa0
do_fast_syscall_32+0xa1/0x1b0
entry_SYSENTER_32+0x4c/0x7b
EIP: 0xb7786c61
EFLAGS:
00000293 CPU: 2
EAX:
ffffffda EBX:
00000003 ECX:
08416000 EDX:
00001000
ESI:
18b24000 EDI:
00000000 EBP:
00000003 ESP:
bf9b36b0
DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
Fixes:
aaec2b1d1879 ("f2fs: introduce cp_lock to protect updating of ckpt_flags")
Cc: stable@vger.kernel.org
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 6 Jul 2017 21:46:01 +0000 (14:46 -0700)]
f2fs: relax migratepage for atomic written page
In order to avoid lock contention for atomic written pages, we'd better give
EBUSY in f2fs_migrate_page when mode is asynchronous. We expect it will be
released soon as transaction commits.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 5 Jul 2017 17:11:31 +0000 (01:11 +0800)]
f2fs: don't count inode block in in-memory inode.i_blocks
Previously, we count all inode consumed blocks including inode block,
xattr block, index block, data block into i_blocks, for other generic
filesystems, they won't count inode block into i_blocks, so for
userspace applications or quota system, they may detect incorrect block
count according to i_blocks value in inode.
This patch changes to count all blocks into inode.i_blocks excluding
inode block, for on-disk i_blocks, we keep counting inode block for
backward compatibility.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>