user/sven/linux.git/fs/f2fs/segment.c, branch v5.4.55

f2fs: Fix deadlock in f2fs_gc() context during atomic files handling

2020-01-04T18:18:18Z

[ Upstream commit 677017d196ba2a4cfff13626b951cc9a206b8c7c ] The FS got stuck in the below stack when the storage is almost full/dirty condition (when FG_GC is being done). schedule_timeout io_schedule_timeout congestion_wait f2fs_drop_inmem_pages_all f2fs_gc f2fs_balance_fs __write_node_page f2fs_fsync_node_pages f2fs_do_sync_file f2fs_ioctl The root cause for this issue is there is a potential infinite loop in f2fs_drop_inmem_pages_all() for the case where gc_failure is true and when there an inode whose i_gc_failures[GC_FAILURE_ATOMIC] is not set. Fix this by keeping track of the total atomic files currently opened and using that to exit from this condition. Fix-suggested-by: Chao Yu Signed-off-by: Chao Yu Signed-off-by: Sahitya Tummala Signed-off-by: Jaegeuk Kim Signed-off-by: Sasha Levin

f2fs: fix to add missing F2FS_IO_ALIGNED() condition

2019-09-16T15:38:49Z

In f2fs_allocate_data_block(), we will reset fio.retry for IO alignment feature instead of IO serialization feature. In addition, spread F2FS_IO_ALIGNED() to check IO alignment feature status explicitly. Signed-off-by: Chao Yu Signed-off-by: Jaegeuk Kim

f2fs: avoid infinite GC loop due to stale atomic files

2019-09-16T15:38:20Z

If committing atomic pages is failed when doing f2fs_do_sync_file(), we can get commited pages but atomic_file being still set like: - inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0) If GC selects this block, we can get an infinite loop like this: f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATA f2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096 f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234 f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATA f2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096 f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234 f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c In that moment, we can observe: [Before] Try to move 5084219 blocks (BG: 384508) - data blocks : 4962373 (274483) - node blocks : 121846 (110025) Skipped : atomic write 4534686 (10) [After] Try to move 5088973 blocks (BG: 384508) - data blocks : 4967127 (274483) - node blocks : 121846 (110025) Skipped : atomic write 4539440 (10) So, refactor atomic_write flow like this: 1. start_atomic_write - add inmem_list and set atomic_file 2. write() - register it in inmem_pages 3. commit_atomic_write - if no error, f2fs_drop_inmem_pages() - f2fs_commit_inmme_pages() failed : __revoked_inmem_pages() was done - f2fs_do_sync_file failed : abort_atomic_write later 4. abort_atomic_write - f2fs_drop_inmem_pages 5. f2fs_drop_inmem_pages - clear atomic_file - remove inmem_list Based on this change, when GC fails to move block in atomic_file, f2fs_drop_inmem_pages_all() can call f2fs_drop_inmem_pages(). Reviewed-by: Chao Yu Signed-off-by: Jaegeuk Kim

f2fs: enhance f2fs_is_checkpoint_ready()'s readability

2019-09-06T23:18:26Z

This patch changes sematics of f2fs_is_checkpoint_ready()'s return value as: return true when checkpoint is ready, other return false, it can improve readability of below conditions. f2fs_submit_page_write() ... if (is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN) || !f2fs_is_checkpoint_ready(sbi)) __submit_merged_bio(io); f2fs_balance_fs() ... if (!f2fs_is_checkpoint_ready(sbi)) return; Signed-off-by: Chao Yu Signed-off-by: Jaegeuk Kim

f2fs: Fix indefinite loop in f2fs_gc()

2019-08-23T14:57:15Z

Policy - Foreground GC, LFS and greedy GC mode. Under this policy, f2fs_gc() loops forever to GC as it doesn't have enough free segements to proceed and thus it keeps calling gc_more for the same victim segment. This can happen if the selected victim segment could not be GC'd due to failed blkaddr validity check i.e. is_alive() returns false for the blocks set in current validity map. Fix this by keeping track of such invalid segments and skip those segments for selection in get_victim_by_default() to avoid endless GC loop under such error scenarios. Currently, add this logic under CONFIG_F2FS_CHECK_FS to be able to root cause the issue in debug version. Signed-off-by: Sahitya Tummala Reviewed-by: Chao Yu [Jaegeuk Kim: fix wrong bitmap size] Signed-off-by: Jaegeuk Kim

f2fs: allocate memory in batch in build_sit_info()

2019-08-23T14:57:15Z

build_sit_info() allocate all bitmaps for each segment one by one, it's quite low efficiency, this pach changes to allocate large continuous memory at a time, and divide it and assign for each bitmaps of segment. For large size image, it can expect improving its mount speed. Signed-off-by: Chen Gong Signed-off-by: Chao Yu Signed-off-by: Jaegeuk Kim

f2fs: fix to avoid data corruption by forbidding SSR overwrite

2019-08-23T14:57:14Z

There is one case can cause data corruption. - write 4k to fileA - fsync fileA, 4k data is writebacked to lbaA - write 4k to fileA - kworker flushs 4k to lbaB; dnode contain lbaB didn't be persisted yet - write 4k to fileB - kworker flush 4k to lbaA due to SSR - SPOR -> dnode with lbaA will be recovered, however lbaA contains fileB's data One solution is tracking all fsynced file's block history, and disallow SSR overwrite on newly invalidated block on that file. However, during recovery, no matter the dnode is flushed or fsynced, all previous dnodes until last fsynced one in node chain can be recovered, that means we need to record all block change in flushed dnode, which will cause heavy cost, so let's just use simple fix by forbidding SSR overwrite directly. Fixes: 5b6c6be2d878 ("f2fs: use SSR for warm node as well") Signed-off-by: Chao Yu Signed-off-by: Jaegeuk Kim

Revert "f2fs: avoid out-of-range memory access"

2019-08-23T14:57:14Z

As Pavel Machek reported: "We normally use -EUCLEAN to signal filesystem corruption. Plus, it is good idea to report it to the syslog and mark filesystem as "needing fsck" if filesystem can do that." Still we need improve the original patch with: - use unlikely keyword - add message print - return EUCLEAN However, after rethink this patch, I don't think we should add such condition check here as below reasons: - We have already checked the field in f2fs_sanity_check_ckpt(), - If there is fs corrupt or security vulnerability, there is nothing to guarantee the field is integrated after the check, unless we do the check before each of its use, however no filesystem does that. - We only have similar check for bitmap, which was added due to there is bitmap corruption happened on f2fs' runtime in product. - There are so many key fields in SB/CP/NAT did have such check after f2fs_sanity_check_{sb,cp,..}. So I propose to revert this unneeded check. This reverts commit 56f3ce675103e3fb9e631cfb4131fc768bc23e9a. Signed-off-by: Chao Yu Signed-off-by: Jaegeuk Kim

f2fs: cleanup the code in build_sit_entries.

2019-08-23T14:57:14Z

We do not need to set the SBI_NEED_FSCK flag in the error paths, if we return error here, we will not update the checkpoint flag, so the code is useless, just remove it. Signed-off-by: Lihong Kou Reviewed-by: Chao Yu Signed-off-by: Jaegeuk Kim

f2fs: fix to avoid discard command leak

2019-08-23T14:57:11Z

============================================================================= BUG discard_cmd (Tainted: G B OE ): Objects remaining in discard_cmd on __kmem_cache_shutdown() ----------------------------------------------------------------------------- INFO: Slab 0xffffe1ac481d22c0 objects=36 used=2 fp=0xffff936b4748bf50 flags=0x2ffff0000000100 Call Trace: dump_stack+0x63/0x87 slab_err+0xa1/0xb0 __kmem_cache_shutdown+0x183/0x390 shutdown_cache+0x14/0x110 kmem_cache_destroy+0x195/0x1c0 f2fs_destroy_segment_manager_caches+0x21/0x40 [f2fs] exit_f2fs_fs+0x35/0x641 [f2fs] SyS_delete_module+0x155/0x230 ? vtime_user_exit+0x29/0x70 do_syscall_64+0x6e/0x160 entry_SYSCALL64_slow_path+0x25/0x25 INFO: Object 0xffff936b4748b000 @offset=0 INFO: Object 0xffff936b4748b070 @offset=112 kmem_cache_destroy discard_cmd: Slab cache still has objects Call Trace: dump_stack+0x63/0x87 kmem_cache_destroy+0x1b4/0x1c0 f2fs_destroy_segment_manager_caches+0x21/0x40 [f2fs] exit_f2fs_fs+0x35/0x641 [f2fs] SyS_delete_module+0x155/0x230 do_syscall_64+0x6e/0x160 entry_SYSCALL64_slow_path+0x25/0x25 Recovery can cache discard commands, so in error path of fill_super(), we need give a chance to handle them, otherwise it will lead to leak of discard_cmd slab cache. Signed-off-by: Chao Yu Signed-off-by: Jaegeuk Kim