From 88c4650a9ece8fef2be042fbbec2dde2d0afa1a4 Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 17 Jun 2002 20:19:13 -0700 Subject: [PATCH] direct-to-BIO I/O for swapcache pages This patch changes the swap I/O handling. The objectives are: - Remove swap special-casing - Stop using buffer_heads -> direct-to-BIO - Make S_ISREG swapfiles more robust. I've spent quite some time with swap. The first patches converted swap to use block_read/write_full_page(). These were discarded because they are still using buffer_heads, and a reasonable amount of otherwise unnecessary infrastructure had to be added to the swap code just to make it look like a regular fs. So this code just has a custom direct-to-BIO path for swap, which seems to be the most comfortable approach. A significant thing here is the introduction of "swap extents". A swap extent is a simple data structure which maps a range of swap pages onto a range of disk sectors. It is simply: struct swap_extent { struct list_head list; pgoff_t start_page; pgoff_t nr_pages; sector_t start_block; }; At swapon time (for an S_ISREG swapfile), each block in the file is bmapped() and the block numbers are parsed to generate the device's swap extent list. This extent list is quite compact - a 512 megabyte swapfile generates about 130 nodes in the list. That's about 4 kbytes of storage. The conversion from filesystem blocksize blocks into PAGE_SIZE blocks is performed at swapon time. At swapon time (for an S_ISBLK swapfile), we install a single swap extent which describes the entire device. The advantages of the swap extents are: 1: We never have to run bmap() (ie: read from disk) at swapout time. So S_ISREG swapfiles are now just as robust as S_ISBLK swapfiles. 2: All the differences between S_ISBLK swapfiles and S_ISREG swapfiles are handled at swapon time. During normal operation, we just don't care. Both types of swapfiles are handled the same way. 3: The extent lists always operate in PAGE_SIZE units. So the problems of going from fs blocksize to PAGE_SIZE are handled at swapon time and normal operating code doesn't need to care. 4: Because we don't have to fiddle with different blocksizes, we can go direct-to-BIO for swap_readpage() and swap_writepage(). This introduces the kernel-wide invariant "anonymous pages never have buffers attached", which cleans some things up nicely. All those block_flushpage() calls in the swap code simply go away. 5: The kernel no longer has to allocate both buffer_heads and BIOs to perform swapout. Just a BIO. 6: It permits us to perform swapcache writeout and throttling for GFP_NOFS allocations (a later patch). (Well, there is one sort of anon page which can have buffers: the pages which are cast adrift in truncate_complete_page() because do_invalidatepage() failed. But these pages are never added to swapcache, and nobody except the VM LRU has to deal with them). The swapfile parser in setup_swap_extents() will attempt to extract the largest possible number of PAGE_SIZE-sized and PAGE_SIZE-aligned chunks of disk from the S_ISREG swapfile. Any stray blocks (due to file discontiguities) are simply discarded - we never swap to those. If an S_ISREG swapfile is found to have any unmapped blocks (file holes) then the swapon attempt will fail. The extent list can be quite large (hundreds of nodes for a gigabyte S_ISREG swapfile). It needs to be consulted once for each page within swap_readpage() and swap_writepage(). Hence there is a risk that we could blow significant amounts of CPU walking that list. However I have implemented a "where we found the last block" cache, which is used as the starting point for the next search. Empirical testing indicates that this is wildly effective - the average length of the list walk in map_swap_page() is 0.3 iterations per page, with a 130-element list. It _could_ be that some workloads do start suffering long walks in that code, and perhaps a tree would be needed there. But I doubt that, and if this is happening then it means that we're seeking all over the disk for swap I/O, and the list walk is the least of our problems. rw_swap_page_nolock() now takes a page*, not a kernel virtual address. It has been renamed to rw_swap_page_sync() and it takes care of locking and unlocking the page itself. Which is all a much better interface. Support for type 0 swap has been removed. Current versions of mkwap(8) seem to never produce v0 swap unless you explicitly ask for it, so I doubt if this will affect anyone. If you _do_ have a type 0 swapfile, swapon will fail and the message version 0 swap is no longer supported. Use mkswap -v1 /dev/sdb3 is printed. We can remove that code for real later on. Really, all that swapfile header parsing should be pushed out to userspace. This code always uses single-page BIOs for swapin and swapout. I have an additional patch which converts swap to use mpage_writepages(), so we swap out in 16-page BIOs. It works fine, but I don't intend to submit that. There just doesn't seem to be any significant advantage to it. I can't see anything in sys_swapon()/sys_swapoff() which needs the lock_kernel() calls, so I deleted them. If you ftruncate an S_ISREG swapfile to a shorter size while it is in use, subsequent swapout will destroy the filesystem. It was always thus, but it is much, much easier to do now. Not really a kernel problem, but swapon(8) should not be allowing the kernel to use swapfiles which are modifiable by unprivileged users. --- include/linux/buffer_head.h | 1 - 1 file changed, 1 deletion(-) (limited to 'include/linux/buffer_head.h') diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 90767fc78617..fda967ab9358 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -183,7 +183,6 @@ struct buffer_head * __bread(struct block_device *, int, int); void wakeup_bdflush(void); struct buffer_head *alloc_buffer_head(int async); void free_buffer_head(struct buffer_head * bh); -int brw_page(int, struct page *, struct block_device *, sector_t [], int); void FASTCALL(unlock_buffer(struct buffer_head *bh)); /* -- cgit v1.2.3 From c67b85b06126b960a44c1dcdf809cf9e947f4a1c Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 17 Jun 2002 20:20:09 -0700 Subject: [PATCH] clean up alloc_buffer_head() alloc_bufer_head() does not need the additional argument - GFP_NOFS is always correct. --- fs/buffer.c | 14 +++++++------- fs/jbd/journal.c | 2 +- include/linux/buffer_head.h | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) (limited to 'include/linux/buffer_head.h') diff --git a/fs/buffer.c b/fs/buffer.c index 5d9dee75f287..70025ee603a6 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -945,12 +945,12 @@ void invalidate_inode_buffers(struct inode *inode) * the size of each buffer.. Use the bh->b_this_page linked list to * follow the buffers created. Return NULL if unable to create more * buffers. - * The async flag is used to differentiate async IO (paging, swapping) - * from ordinary buffer allocations, and only async requests are allowed - * to sleep waiting for buffer heads. + * + * The retry flag is used to differentiate async IO (paging, swapping) + * which may not fail from ordinary buffer allocations. */ static struct buffer_head * -create_buffers(struct page * page, unsigned long size, int async) +create_buffers(struct page * page, unsigned long size, int retry) { struct buffer_head *bh, *head; long offset; @@ -959,7 +959,7 @@ try_again: head = NULL; offset = PAGE_SIZE; while ((offset -= size) >= 0) { - bh = alloc_buffer_head(async); + bh = alloc_buffer_head(); if (!bh) goto no_grow; @@ -996,7 +996,7 @@ no_grow: * become available. But we don't want tasks sleeping with * partially complete buffers, so all were released above. */ - if (!async) + if (!retry) return NULL; /* We're _really_ low on memory. Now we just @@ -2392,7 +2392,7 @@ asmlinkage long sys_bdflush(int func, long data) static kmem_cache_t *bh_cachep; static mempool_t *bh_mempool; -struct buffer_head *alloc_buffer_head(int async) +struct buffer_head *alloc_buffer_head(void) { return mempool_alloc(bh_mempool, GFP_NOFS); } diff --git a/fs/jbd/journal.c b/fs/jbd/journal.c index 052dd4ef3f01..ade37ad43606 100644 --- a/fs/jbd/journal.c +++ b/fs/jbd/journal.c @@ -463,7 +463,7 @@ int journal_write_metadata_buffer(transaction_t *transaction, * Right, time to make up the new buffer_head. */ do { - new_bh = alloc_buffer_head(0); + new_bh = alloc_buffer_head(); if (!new_bh) { printk (KERN_NOTICE "%s: ENOMEM at alloc_buffer_head, " "trying again.\n", __FUNCTION__); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index fda967ab9358..5df63727fa7a 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -181,7 +181,7 @@ void __brelse(struct buffer_head *); void __bforget(struct buffer_head *); struct buffer_head * __bread(struct block_device *, int, int); void wakeup_bdflush(void); -struct buffer_head *alloc_buffer_head(int async); +struct buffer_head *alloc_buffer_head(void); void free_buffer_head(struct buffer_head * bh); void FASTCALL(unlock_buffer(struct buffer_head *bh)); -- cgit v1.2.3 From a28b4d4ede7e5f9bcde157417957998571b7a639 Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 17 Jun 2002 20:20:24 -0700 Subject: [PATCH] take bio.h out of highmem.h highmem.h includes bio.h, so just about every compilation unit in the kernel gets to process bio.h. The patch moves the BIO-related functions out of highmem.h and into bio-related headers. The nested include is removed and all files which need to include bio.h now do so. --- drivers/block/DAC960.c | 1 + drivers/block/cciss.c | 1 + drivers/block/cpqarray.c | 1 + drivers/block/elevator.c | 1 + drivers/block/floppy.c | 1 + drivers/block/ll_rw_blk.c | 1 + drivers/block/loop.c | 1 + drivers/block/nbd.c | 1 + drivers/block/rd.c | 2 ++ drivers/block/umem.c | 1 + drivers/md/linear.c | 2 +- drivers/md/lvm.c | 1 + drivers/md/md.c | 1 + drivers/md/multipath.c | 1 + drivers/md/raid0.c | 1 + drivers/md/raid1.c | 1 + drivers/md/raid5.c | 1 + drivers/scsi/cpqfcTSinit.c | 1 + drivers/scsi/scsi_lib.c | 1 + drivers/scsi/sd.c | 1 + drivers/scsi/sr.c | 1 + fs/bio.c | 1 + fs/jfs/jfs_logmgr.c | 1 + include/linux/bio.h | 50 ++++++++++++++++++++++++++++++++++++++++++--- include/linux/blkdev.h | 7 +------ include/linux/buffer_head.h | 5 ----- include/linux/highmem.h | 44 --------------------------------------- include/linux/ide.h | 1 + include/linux/raid/raid5.h | 1 + kernel/ksyms.c | 2 +- mm/highmem.c | 5 ++++- 31 files changed, 79 insertions(+), 61 deletions(-) (limited to 'include/linux/buffer_head.h') diff --git a/drivers/block/DAC960.c b/drivers/block/DAC960.c index d57dc51df3f5..210449ad1715 100644 --- a/drivers/block/DAC960.c +++ b/drivers/block/DAC960.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c index 9ae961460ff2..e06fd274b653 100644 --- a/drivers/block/cciss.c +++ b/drivers/block/cciss.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/cpqarray.c b/drivers/block/cpqarray.c index 727cdeb23c0c..fccef1bb792c 100644 --- a/drivers/block/cpqarray.c +++ b/drivers/block/cpqarray.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/elevator.c b/drivers/block/elevator.c index 189814dbc7d1..cd3a4254e9e3 100644 --- a/drivers/block/elevator.c +++ b/drivers/block/elevator.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c index 94f42b356556..aff8acff0ef3 100644 --- a/drivers/block/floppy.c +++ b/drivers/block/floppy.c @@ -165,6 +165,7 @@ static int print_unex=1; #include #include #include +#include #include #include #include diff --git a/drivers/block/ll_rw_blk.c b/drivers/block/ll_rw_blk.c index 3527afa3cae7..16abcb3f5481 100644 --- a/drivers/block/ll_rw_blk.c +++ b/drivers/block/ll_rw_blk.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/loop.c b/drivers/block/loop.c index ecc87aa5a819..982604ff6bfd 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -60,6 +60,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 67344c7fcc1a..697e825c3a91 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -39,6 +39,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/block/rd.c b/drivers/block/rd.c index 4faf52c7be5c..7b60e75d5584 100644 --- a/drivers/block/rd.c +++ b/drivers/block/rd.c @@ -45,6 +45,8 @@ #include #include #include +#include +#include #include #include #include diff --git a/drivers/block/umem.c b/drivers/block/umem.c index 8c61688cab1c..44909021aa06 100644 --- a/drivers/block/umem.c +++ b/drivers/block/umem.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/md/linear.c b/drivers/md/linear.c index 118ce821a208..48fb74e50d5c 100644 --- a/drivers/md/linear.c +++ b/drivers/md/linear.c @@ -20,7 +20,7 @@ #include #include - +#include #include #define MAJOR_NR MD_MAJOR diff --git a/drivers/md/lvm.c b/drivers/md/lvm.c index dfc256c6a2ec..c44a1b8a74b2 100644 --- a/drivers/md/lvm.c +++ b/drivers/md/lvm.c @@ -209,6 +209,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/md/md.c b/drivers/md/md.c index 21e20ea10be7..d23270322804 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include diff --git a/drivers/md/multipath.c b/drivers/md/multipath.c index 46f089ee8481..6db555317b13 100644 --- a/drivers/md/multipath.c +++ b/drivers/md/multipath.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c index 430448c566af..8f149a1efe1b 100644 --- a/drivers/md/raid0.c +++ b/drivers/md/raid0.c @@ -20,6 +20,7 @@ #include #include +#include #define MAJOR_NR MD_MAJOR #define MD_DRIVER diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 43fdb75de0fe..96ad858cf033 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -23,6 +23,7 @@ */ #include +#include #define MAJOR_NR MD_MAJOR #define MD_DRIVER diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 9402b0c779b9..62873d89e395 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include diff --git a/drivers/scsi/cpqfcTSinit.c b/drivers/scsi/cpqfcTSinit.c index e6f03847c212..f38e377207c7 100644 --- a/drivers/scsi/cpqfcTSinit.c +++ b/drivers/scsi/cpqfcTSinit.c @@ -39,6 +39,7 @@ #include #include #include +#include #include // request_region() prototype #include // ioremap() //#if LINUX_VERSION_CODE >= LinuxVersionCode(2,4,7) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index fc69760ab484..bede96547efb 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 382e04ceace2..63fe305e4342 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c index d536f3bc94f6..0e28dc69652b 100644 --- a/drivers/scsi/sr.c +++ b/drivers/scsi/sr.c @@ -39,6 +39,7 @@ #include #include #include +#include #include #include #include diff --git a/fs/bio.c b/fs/bio.c index 00cd91f8aaa1..5fdae32e35ae 100644 --- a/fs/bio.c +++ b/fs/bio.c @@ -17,6 +17,7 @@ * */ #include +#include #include #include #include diff --git a/fs/jfs/jfs_logmgr.c b/fs/jfs/jfs_logmgr.c index ea37f1c39a64..7790f413096a 100644 --- a/fs/jfs/jfs_logmgr.c +++ b/fs/jfs/jfs_logmgr.c @@ -65,6 +65,7 @@ #include #include #include /* for sync_blockdev() */ +#include #include "jfs_incore.h" #include "jfs_filsys.h" #include "jfs_metapage.h" diff --git a/include/linux/bio.h b/include/linux/bio.h index b244108a27a8..ffc38fca9c1e 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -21,6 +21,8 @@ #define __LINUX_BIO_H #include +#include + /* Platforms may set this to teach the BIO layer about IOMMU hardware. */ #include #ifndef BIO_VMERGE_BOUNDARY @@ -47,9 +49,6 @@ struct bio_vec { unsigned int bv_offset; }; -/* - * weee, c forward decl... - */ struct bio; typedef void (bio_end_io_t) (struct bio *); typedef void (bio_destructor_t) (struct bio *); @@ -206,4 +205,49 @@ extern inline void bio_init(struct bio *); extern int bio_ioctl(kdev_t, unsigned int, unsigned long); +#ifdef CONFIG_HIGHMEM +/* + * remember to add offset! and never ever reenable interrupts between a + * bio_kmap_irq and bio_kunmap_irq!! + * + * This function MUST be inlined - it plays with the CPU interrupt flags. + * Hence the `extern inline'. + */ +extern inline char *bio_kmap_irq(struct bio *bio, unsigned long *flags) +{ + unsigned long addr; + + __save_flags(*flags); + + /* + * could be low + */ + if (!PageHighMem(bio_page(bio))) + return bio_data(bio); + + /* + * it's a highmem page + */ + __cli(); + addr = (unsigned long) kmap_atomic(bio_page(bio), KM_BIO_SRC_IRQ); + + if (addr & ~PAGE_MASK) + BUG(); + + return (char *) addr + bio_offset(bio); +} + +extern inline void bio_kunmap_irq(char *buffer, unsigned long *flags) +{ + unsigned long ptr = (unsigned long) buffer & PAGE_MASK; + + kunmap_atomic((void *) ptr, KM_BIO_SRC_IRQ); + __restore_flags(*flags); +} + +#else +#define bio_kmap_irq(bio, flags) (bio_data(bio)) +#define bio_kunmap_irq(buf, flags) do { *(flags) = 0; } while (0) +#endif + #endif /* __LINUX_BIO_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index ef86a3ed6e64..c0c099834df2 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -246,12 +246,7 @@ extern unsigned long blk_max_low_pfn, blk_max_pfn; #define BLK_BOUNCE_ISA (ISA_DMA_THRESHOLD) extern int init_emergency_isa_pool(void); -extern void create_bounce(unsigned long pfn, int gfp, struct bio **bio_orig); - -extern inline void blk_queue_bounce(request_queue_t *q, struct bio **bio) -{ - create_bounce(q->bounce_pfn, q->bounce_gfp, bio); -} +void blk_queue_bounce(request_queue_t *q, struct bio **bio); #define rq_for_each_bio(bio, rq) \ if ((rq->bio)) \ diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 5df63727fa7a..96207c0d6dce 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -108,12 +108,7 @@ BUFFER_FNS(Async_Read, async_read) BUFFER_FNS(Async_Write, async_write) BUFFER_FNS(Boundary, boundary) -/* - * FIXME: this is used only by bh_kmap, which is used only by RAID5. - * Move all that stuff into raid5.c - */ #define bh_offset(bh) ((unsigned long)(bh)->b_data & ~PAGE_MASK) - #define touch_buffer(bh) mark_page_accessed(bh->b_page) /* If we *know* page->private refers to buffer_heads */ diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 54fb9176fb75..68c841afc622 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -2,7 +2,6 @@ #define _LINUX_HIGHMEM_H #include -#include #include #include @@ -15,45 +14,8 @@ extern struct page *highmem_start_page; /* declarations for linux/mm/highmem.c */ unsigned int nr_free_highpages(void); -extern void create_bounce(unsigned long pfn, int gfp, struct bio **bio_orig); extern void check_highmem_ptes(void); -/* - * remember to add offset! and never ever reenable interrupts between a - * bio_kmap_irq and bio_kunmap_irq!! - */ -static inline char *bio_kmap_irq(struct bio *bio, unsigned long *flags) -{ - unsigned long addr; - - __save_flags(*flags); - - /* - * could be low - */ - if (!PageHighMem(bio_page(bio))) - return bio_data(bio); - - /* - * it's a highmem page - */ - __cli(); - addr = (unsigned long) kmap_atomic(bio_page(bio), KM_BIO_SRC_IRQ); - - if (addr & ~PAGE_MASK) - BUG(); - - return (char *) addr + bio_offset(bio); -} - -static inline void bio_kunmap_irq(char *buffer, unsigned long *flags) -{ - unsigned long ptr = (unsigned long) buffer & PAGE_MASK; - - kunmap_atomic((void *) ptr, KM_BIO_SRC_IRQ); - __restore_flags(*flags); -} - #else /* CONFIG_HIGHMEM */ static inline unsigned int nr_free_highpages(void) { return 0; } @@ -65,12 +27,6 @@ static inline void *kmap(struct page *page) { return page_address(page); } #define kmap_atomic(page,idx) kmap(page) #define kunmap_atomic(page,idx) kunmap(page) -#define bh_kmap(bh) ((bh)->b_data) -#define bh_kunmap(bh) do { } while (0) - -#define bio_kmap_irq(bio, flags) (bio_data(bio)) -#define bio_kunmap_irq(buf, flags) do { *(flags) = 0; } while (0) - #endif /* CONFIG_HIGHMEM */ /* when CONFIG_HIGHMEM is not set these will be plain clear/copy_page */ diff --git a/include/linux/ide.h b/include/linux/ide.h index e07d0f19fcd1..03c21c567ce4 100644 --- a/include/linux/ide.h +++ b/include/linux/ide.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include diff --git a/include/linux/raid/raid5.h b/include/linux/raid/raid5.h index 5c25120581a7..67f7bf471798 100644 --- a/include/linux/raid/raid5.h +++ b/include/linux/raid/raid5.h @@ -3,6 +3,7 @@ #include #include +#include /* * diff --git a/kernel/ksyms.c b/kernel/ksyms.c index 34a50cd558e9..4e554c38c7f5 100644 --- a/kernel/ksyms.c +++ b/kernel/ksyms.c @@ -120,7 +120,7 @@ EXPORT_SYMBOL(vmtruncate); EXPORT_SYMBOL(find_vma); EXPORT_SYMBOL(get_unmapped_area); EXPORT_SYMBOL(init_mm); -EXPORT_SYMBOL(create_bounce); +EXPORT_SYMBOL(blk_queue_bounce); #ifdef CONFIG_HIGHMEM EXPORT_SYMBOL(kmap_high); EXPORT_SYMBOL(kunmap_high); diff --git a/mm/highmem.c b/mm/highmem.c index de5ebeb0a167..ae9c5a26376b 100644 --- a/mm/highmem.c +++ b/mm/highmem.c @@ -17,6 +17,7 @@ */ #include +#include #include #include #include @@ -347,13 +348,15 @@ static void bounce_end_io_read_isa(struct bio *bio) return __bounce_end_io_read(bio, isa_page_pool); } -void create_bounce(unsigned long pfn, int gfp, struct bio **bio_orig) +void blk_queue_bounce(request_queue_t *q, struct bio **bio_orig) { struct page *page; struct bio *bio = NULL; int i, rw = bio_data_dir(*bio_orig), bio_gfp; struct bio_vec *to, *from; mempool_t *pool; + unsigned long pfn = q->bounce_pfn; + int gfp = q->bounce_gfp; BUG_ON((*bio_orig)->bi_idx); -- cgit v1.2.3 From 38cb52ca07ab22719026ffdfe1db794ed8e9b73b Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 17 Jun 2002 20:20:38 -0700 Subject: [PATCH] remove set_page_buffers() and clear_page_buffers() The set_page_buffers() and clear_page_buffers() macros are each used in only one place. Fold them into their callers. --- fs/buffer.c | 6 ++++-- include/linux/buffer_head.h | 10 ---------- 2 files changed, 4 insertions(+), 12 deletions(-) (limited to 'include/linux/buffer_head.h') diff --git a/fs/buffer.c b/fs/buffer.c index 70025ee603a6..a47863b3bd8a 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -152,14 +152,16 @@ __set_page_buffers(struct page *page, struct buffer_head *head) { if (page_has_buffers(page)) buffer_error(); - set_page_buffers(page, head); page_cache_get(page); + SetPagePrivate(page); + page->private = (unsigned long)head; } static inline void __clear_page_buffers(struct page *page) { - clear_page_buffers(page); + ClearPagePrivate(page); + page->private = 0; page_cache_release(page); } diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 96207c0d6dce..903850460ab5 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -119,16 +119,6 @@ BUFFER_FNS(Boundary, boundary) ((struct buffer_head *)(page)->private); \ }) #define page_has_buffers(page) PagePrivate(page) -#define set_page_buffers(page, buffers) \ - do { \ - SetPagePrivate(page); \ - page->private = (unsigned long)buffers; \ - } while (0) -#define clear_page_buffers(page) \ - do { \ - ClearPagePrivate(page); \ - page->private = 0; \ - } while (0) #define invalidate_buffers(dev) __invalidate_buffers((dev), 0) #define destroy_buffers(dev) __invalidate_buffers((dev), 1) -- cgit v1.2.3 From 3fb3b749ba712dd4197b585d654d233d3487d9d4 Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 17 Jun 2002 20:21:09 -0700 Subject: [PATCH] rename get_hash_table() to find_get_block() Renames the buffer_head lookup function `get_hash_table' to `find_get_block'. get_hash_table() is too generic a name. Plus it doesn't even use a hash any more. --- drivers/md/lvm-snap.c | 2 +- fs/buffer.c | 10 +++++----- fs/ext3/balloc.c | 4 ++-- fs/ext3/inode.c | 2 +- fs/jbd/revoke.c | 6 +++--- fs/qnx4/fsync.c | 2 +- fs/reiserfs/fix_node.c | 2 +- fs/reiserfs/journal.c | 8 ++++---- fs/ufs/truncate.c | 8 ++++---- include/linux/buffer_head.h | 6 +++--- include/linux/reiserfs_fs.h | 2 +- kernel/ksyms.c | 2 +- 12 files changed, 27 insertions(+), 27 deletions(-) (limited to 'include/linux/buffer_head.h') diff --git a/drivers/md/lvm-snap.c b/drivers/md/lvm-snap.c index c90947fc5f89..46df5c8ff0ef 100644 --- a/drivers/md/lvm-snap.c +++ b/drivers/md/lvm-snap.c @@ -224,7 +224,7 @@ static inline void invalidate_snap_cache(unsigned long start, unsigned long nr, for (i = 0; i < nr; i++) { - bh = get_hash_table(dev, start++, blksize); + bh = find_get_block(dev, start++, blksize); if (bh) bforget(bh); } diff --git a/fs/buffer.c b/fs/buffer.c index a47863b3bd8a..dde8e7d9bae6 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -378,7 +378,7 @@ out: } /* - * Various filesystems appear to want __get_hash_table to be non-blocking. + * Various filesystems appear to want __find_get_block to be non-blocking. * But it's the page lock which protects the buffers. To get around this, * we get exclusion from try_to_free_buffers with the blockdev mapping's * private_lock. @@ -389,7 +389,7 @@ out: * private_lock is contended then so is mapping->page_lock). */ struct buffer_head * -__get_hash_table(struct block_device *bdev, sector_t block, int unused) +__find_get_block(struct block_device *bdev, sector_t block, int unused) { struct inode *bd_inode = bdev->bd_inode; struct address_space *bd_mapping = bd_inode->i_mapping; @@ -1091,7 +1091,7 @@ grow_dev_page(struct block_device *bdev, unsigned long block, /* * Link the page to the buffers and initialise them. Take the - * lock to be atomic wrt __get_hash_table(), which does not + * lock to be atomic wrt __find_get_block(), which does not * run under the page lock. */ spin_lock(&inode->i_mapping->private_lock); @@ -1164,7 +1164,7 @@ __getblk(struct block_device *bdev, sector_t block, int size) for (;;) { struct buffer_head * bh; - bh = __get_hash_table(bdev, block, size); + bh = __find_get_block(bdev, block, size); if (bh) { touch_buffer(bh); return bh; @@ -1449,7 +1449,7 @@ void unmap_underlying_metadata(struct block_device *bdev, sector_t block) { struct buffer_head *old_bh; - old_bh = __get_hash_table(bdev, block, 0); + old_bh = __find_get_block(bdev, block, 0); if (old_bh) { #if 0 /* This happens. Later. */ if (buffer_dirty(old_bh)) diff --git a/fs/ext3/balloc.c b/fs/ext3/balloc.c index f8f6828d5f59..c5cc2178ad4a 100644 --- a/fs/ext3/balloc.c +++ b/fs/ext3/balloc.c @@ -352,7 +352,7 @@ do_more: #ifdef CONFIG_JBD_DEBUG { struct buffer_head *debug_bh; - debug_bh = sb_get_hash_table(sb, block + i); + debug_bh = sb_find_get_block(sb, block + i); if (debug_bh) { BUFFER_TRACE(debug_bh, "Deleted!"); if (!bh2jh(bitmap_bh)->b_committed_data) @@ -701,7 +701,7 @@ got_block: struct buffer_head *debug_bh; /* Record bitmap buffer state in the newly allocated block */ - debug_bh = sb_get_hash_table(sb, tmp); + debug_bh = sb_find_get_block(sb, tmp); if (debug_bh) { BUFFER_TRACE(debug_bh, "state when allocated"); BUFFER_TRACE2(debug_bh, bh, "bitmap state"); diff --git a/fs/ext3/inode.c b/fs/ext3/inode.c index b339c253628e..a9b2c7beb70b 100644 --- a/fs/ext3/inode.c +++ b/fs/ext3/inode.c @@ -1650,7 +1650,7 @@ ext3_clear_blocks(handle_t *handle, struct inode *inode, struct buffer_head *bh, struct buffer_head *bh; *p = 0; - bh = sb_get_hash_table(inode->i_sb, nr); + bh = sb_find_get_block(inode->i_sb, nr); ext3_forget(handle, 0, inode, bh, nr); } } diff --git a/fs/jbd/revoke.c b/fs/jbd/revoke.c index 7cecb0237988..6a6464533c35 100644 --- a/fs/jbd/revoke.c +++ b/fs/jbd/revoke.c @@ -293,7 +293,7 @@ int journal_revoke(handle_t *handle, unsigned long blocknr, bh = bh_in; if (!bh) { - bh = __get_hash_table(bdev, blocknr, journal->j_blocksize); + bh = __find_get_block(bdev, blocknr, journal->j_blocksize); if (bh) BUFFER_TRACE(bh, "found on hash"); } @@ -303,7 +303,7 @@ int journal_revoke(handle_t *handle, unsigned long blocknr, /* If there is a different buffer_head lying around in * memory anywhere... */ - bh2 = __get_hash_table(bdev, blocknr, journal->j_blocksize); + bh2 = __find_get_block(bdev, blocknr, journal->j_blocksize); if (bh2) { /* ... and it has RevokeValid status... */ if ((bh2 != bh) && @@ -407,7 +407,7 @@ int journal_cancel_revoke(handle_t *handle, struct journal_head *jh) * state machine will get very upset later on. */ if (need_cancel) { struct buffer_head *bh2; - bh2 = __get_hash_table(bh->b_bdev, bh->b_blocknr, bh->b_size); + bh2 = __find_get_block(bh->b_bdev, bh->b_blocknr, bh->b_size); if (bh2) { if (bh2 != bh) clear_bit(BH_Revoked, &bh2->b_state); diff --git a/fs/qnx4/fsync.c b/fs/qnx4/fsync.c index 2bb315473ee6..df5bc75d5414 100644 --- a/fs/qnx4/fsync.c +++ b/fs/qnx4/fsync.c @@ -37,7 +37,7 @@ static int sync_block(struct inode *inode, unsigned short *block, int wait) if (!*block) return 0; tmp = *block; - bh = sb_get_hash_table(inode->i_sb, *block); + bh = sb_find_get_block(inode->i_sb, *block); if (!bh) return 0; if (*block != tmp) { diff --git a/fs/reiserfs/fix_node.c b/fs/reiserfs/fix_node.c index 0bdb34c5acf4..1cdcd39a06bd 100644 --- a/fs/reiserfs/fix_node.c +++ b/fs/reiserfs/fix_node.c @@ -920,7 +920,7 @@ static int is_left_neighbor_in_cache( /* Get left neighbor block number. */ n_left_neighbor_blocknr = B_N_CHILD_NUM(p_s_tb->FL[n_h], n_left_neighbor_position); /* Look for the left neighbor in the cache. */ - if ( (left = sb_get_hash_table(p_s_sb, n_left_neighbor_blocknr)) ) { + if ( (left = sb_find_get_block(p_s_sb, n_left_neighbor_blocknr)) ) { RFALSE( buffer_uptodate (left) && ! B_IS_IN_TREE(left), "vs-8170: left neighbor (%b %z) is not in the tree", left, left); diff --git a/fs/reiserfs/journal.c b/fs/reiserfs/journal.c index c16dbdc12ca6..2cf16631e224 100644 --- a/fs/reiserfs/journal.c +++ b/fs/reiserfs/journal.c @@ -689,7 +689,7 @@ retry: count = 0 ; for (i = 0 ; atomic_read(&(jl->j_commit_left)) > 1 && i < (jl->j_len + 1) ; i++) { /* everything but commit_bh */ bn = SB_ONDISK_JOURNAL_1st_BLOCK(s) + (jl->j_start+i) % SB_ONDISK_JOURNAL_SIZE(s); - tbh = journal_get_hash_table(s, bn) ; + tbh = journal_find_get_block(s, bn) ; /* kill this sanity check */ if (count > (orig_commit_left + 2)) { @@ -718,7 +718,7 @@ reiserfs_panic(s, "journal-539: flush_commit_list: BAD count(%d) > orig_commit_l for (i = 0 ; atomic_read(&(jl->j_commit_left)) > 1 && i < (jl->j_len + 1) ; i++) { /* everything but commit_bh */ bn = SB_ONDISK_JOURNAL_1st_BLOCK(s) + (jl->j_start + i) % SB_ONDISK_JOURNAL_SIZE(s) ; - tbh = journal_get_hash_table(s, bn) ; + tbh = journal_find_get_block(s, bn) ; wait_on_buffer(tbh) ; if (!buffer_uptodate(tbh)) { @@ -2764,7 +2764,7 @@ int journal_mark_freed(struct reiserfs_transaction_handle *th, struct super_bloc int cleaned = 0 ; if (reiserfs_dont_log(th->t_super)) { - bh = sb_get_hash_table(p_s_sb, blocknr) ; + bh = sb_find_get_block(p_s_sb, blocknr) ; if (bh && buffer_dirty (bh)) { printk ("journal_mark_freed(dont_log): dirty buffer on hash list: %lx %ld\n", bh->b_state, blocknr); BUG (); @@ -2772,7 +2772,7 @@ int journal_mark_freed(struct reiserfs_transaction_handle *th, struct super_bloc brelse (bh); return 0 ; } - bh = sb_get_hash_table(p_s_sb, blocknr) ; + bh = sb_find_get_block(p_s_sb, blocknr) ; /* if it is journal new, we just remove it from this transaction */ if (bh && buffer_journal_new(bh)) { mark_buffer_notjournal_new(bh) ; diff --git a/fs/ufs/truncate.c b/fs/ufs/truncate.c index f8134d41d98e..6b87c6f26702 100644 --- a/fs/ufs/truncate.c +++ b/fs/ufs/truncate.c @@ -117,7 +117,7 @@ static int ufs_trunc_direct (struct inode * inode) frag1 = ufs_fragnum (frag1); frag2 = ufs_fragnum (frag2); for (j = frag1; j < frag2; j++) { - bh = sb_get_hash_table (sb, tmp + j); + bh = sb_find_get_block (sb, tmp + j); if ((bh && DATA_BUFFER_USED(bh)) || tmp != fs32_to_cpu(sb, *p)) { retry = 1; brelse (bh); @@ -140,7 +140,7 @@ next1: if (!tmp) continue; for (j = 0; j < uspi->s_fpb; j++) { - bh = sb_get_hash_table(sb, tmp + j); + bh = sb_find_get_block(sb, tmp + j); if ((bh && DATA_BUFFER_USED(bh)) || tmp != fs32_to_cpu(sb, *p)) { retry = 1; brelse (bh); @@ -179,7 +179,7 @@ next2:; ufs_panic(sb, "ufs_truncate_direct", "internal error"); frag4 = ufs_fragnum (frag4); for (j = 0; j < frag4; j++) { - bh = sb_get_hash_table (sb, tmp + j); + bh = sb_find_get_block (sb, tmp + j); if ((bh && DATA_BUFFER_USED(bh)) || tmp != fs32_to_cpu(sb, *p)) { retry = 1; brelse (bh); @@ -238,7 +238,7 @@ static int ufs_trunc_indirect (struct inode * inode, unsigned offset, u32 * p) if (!tmp) continue; for (j = 0; j < uspi->s_fpb; j++) { - bh = sb_get_hash_table(sb, tmp + j); + bh = sb_find_get_block(sb, tmp + j); if ((bh && DATA_BUFFER_USED(bh)) || tmp != fs32_to_cpu(sb, *ind)) { retry = 1; brelse (bh); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 903850460ab5..4fc6bab55825 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -160,7 +160,7 @@ int fsync_dev(kdev_t); int fsync_bdev(struct block_device *); int fsync_super(struct super_block *); int fsync_no_super(struct block_device *); -struct buffer_head *__get_hash_table(struct block_device *, sector_t, int); +struct buffer_head *__find_get_block(struct block_device *, sector_t, int); struct buffer_head * __getblk(struct block_device *, sector_t, int); void __brelse(struct buffer_head *); void __bforget(struct buffer_head *); @@ -254,9 +254,9 @@ static inline struct buffer_head * sb_getblk(struct super_block *sb, int block) } static inline struct buffer_head * -sb_get_hash_table(struct super_block *sb, int block) +sb_find_get_block(struct super_block *sb, int block) { - return __get_hash_table(sb->s_bdev, block, sb->s_blocksize); + return __find_get_block(sb->s_bdev, block, sb->s_blocksize); } static inline void diff --git a/include/linux/reiserfs_fs.h b/include/linux/reiserfs_fs.h index 4a3d16d7b8dc..29f6063b3546 100644 --- a/include/linux/reiserfs_fs.h +++ b/include/linux/reiserfs_fs.h @@ -1651,7 +1651,7 @@ extern wait_queue_head_t reiserfs_commit_thread_wait ; #define JOURNAL_BUFFER(j,n) ((j)->j_ap_blocks[((j)->j_start + (n)) % JOURNAL_BLOCK_COUNT]) // We need these to make journal.c code more readable -#define journal_get_hash_table(s, block) __get_hash_table(SB_JOURNAL(s)->j_dev_bd, block, s->s_blocksize) +#define journal_find_get_block(s, block) __find_get_block(SB_JOURNAL(s)->j_dev_bd, block, s->s_blocksize) #define journal_getblk(s, block) __getblk(SB_JOURNAL(s)->j_dev_bd, block, s->s_blocksize) #define journal_bread(s, block) __bread(SB_JOURNAL(s)->j_dev_bd, block, s->s_blocksize) diff --git a/kernel/ksyms.c b/kernel/ksyms.c index 4e554c38c7f5..8b2511787ccb 100644 --- a/kernel/ksyms.c +++ b/kernel/ksyms.c @@ -551,7 +551,7 @@ EXPORT_SYMBOL(file_fsync); EXPORT_SYMBOL(fsync_buffers_list); EXPORT_SYMBOL(clear_inode); EXPORT_SYMBOL(init_special_inode); -EXPORT_SYMBOL(__get_hash_table); +EXPORT_SYMBOL(__find_get_block); EXPORT_SYMBOL(new_inode); EXPORT_SYMBOL(__insert_inode_hash); EXPORT_SYMBOL(remove_inode_hash); -- cgit v1.2.3