Comments on: Page alignment on SSDs http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/ Just another WordPress weblog Sat, 06 Mar 2010 12:56:20 +0000 http://wordpress.org/?v=2.8.2 hourly 1 By: RethinkDB - The database for solid state drives. http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-6944 RethinkDB - The database for solid state drives. Tue, 20 Oct 2009 10:20:24 +0000 http://www.rethinkdb.com/blog/?p=223#comment-6944 [...] Page alignment on SSDs [...] [...] Page alignment on SSDs [...]

]]> By: Slava http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-6663 Slava Mon, 19 Oct 2009 10:30:32 +0000 http://www.rethinkdb.com/blog/?p=223#comment-6663 In this context stride is simply the alignment of the read offset. So, if the stride is 4k, all reads will be performed at a 4k boundary (i.e. an offset into the drive divisible by 4096). In this context stride is simply the alignment of the read offset. So, if the stride is 4k, all reads will be performed at a 4k boundary (i.e. an offset into the drive divisible by 4096).

]]> By: Gabor http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-5878 Gabor Fri, 16 Oct 2009 02:30:37 +0000 http://www.rethinkdb.com/blog/?p=223#comment-5878 Great blog guys! I'm not familiar with the term "stride" in the context of SSDs. I thought stride was the size of an individual chunk striped acruss multiple RAID disks. What does stride mean in the context of DBs on SSDs? Thanks, Gabor Great blog guys! I’m not familiar with the term “stride” in the context of SSDs.

I thought stride was the size of an individual chunk striped acruss multiple RAID disks.

What does stride mean in the context of DBs on SSDs?

Thanks, Gabor

]]> By: Brian Smith http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-5126 Brian Smith Sat, 10 Oct 2009 03:09:24 +0000 http://www.rethinkdb.com/blog/?p=223#comment-5126 I read that minimum_io_size and others were added in 2.6.31. I'm not sure how you retrieve those values in previous kernels. [1] http://www.redhat.com/archives/dm-devel/2009-June/msg00297.html I read that minimum_io_size and others were added in 2.6.31. I’m not sure how you retrieve those values in previous kernels.

[1] http://www.redhat.com/archives/dm-devel/2009-June/msg00297.html

]]> By: Slava http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-5094 Slava Fri, 09 Oct 2009 23:39:50 +0000 http://www.rethinkdb.com/blog/?p=223#comment-5094 Brian, which kernel version are you using? minimum_io_size isn't exposed by sysfs on our 2.6.28-14-server #47-Ubuntu SMP UTC 2009 x86_64 (at least it's not in /sys/block/sdb/queue/minimum_io_size). You're right about the non-zero origin, it can be a bit confusing. You get a slightly better resolution this way, so we chose to use a more close range to present the data. Brian, which kernel version are you using? minimum_io_size isn’t exposed by sysfs on our 2.6.28-14-server #47-Ubuntu SMP UTC 2009 x86_64 (at least it’s not in /sys/block/sdb/queue/minimum_io_size).

You’re right about the non-zero origin, it can be a bit confusing. You get a slightly better resolution this way, so we chose to use a more close range to present the data.

]]> By: Brian Smith http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-5004 Brian Smith Fri, 09 Oct 2009 14:34:18 +0000 http://www.rethinkdb.com/blog/?p=223#comment-5004 I woke up this morning and realized that what I wrote above didn't really make sense. First of all, SSDs can read individual NAND pages--now usually 4K, in the near future 8K--even though they cannot write individual pages. So, the block size shouldn't really matter much. The first thing I would check what Linux is using as minimum_io_size for the drive. If minimum_io_size is 512 then that would explain why 512-byte reads are faster than 4KB reads. [1] That shouldn't be the case because there's no reason (AFAICT) for a SSD to return a minimum_io_size less than 4096, even in 512/4096 emulation mode [1]. The second graph has a non-zero origin. So, graphically it looks like 4K reads are twice as slow as 512 byte reads, but really they are only ~20% slower. And, all SSD drive makers (AFAIK) recommend doing all I/O in blocks the size of a NAND page (4K or 8K). I'd really like to see why there seems to be this 20% performance difference. I look forward to seeing your future posts with results for server-class SSDs. [1] http://mkp.net/pubs/storage-topology.pdf I woke up this morning and realized that what I wrote above didn’t really make sense. First of all, SSDs can read individual NAND pages–now usually 4K, in the near future 8K–even though they cannot write individual pages. So, the block size shouldn’t really matter much.

The first thing I would check what Linux is using as minimum_io_size for the drive. If minimum_io_size is 512 then that would explain why 512-byte reads are faster than 4KB reads. [1] That shouldn’t be the case because there’s no reason (AFAICT) for a SSD to return a minimum_io_size less than 4096, even in 512/4096 emulation mode [1].

The second graph has a non-zero origin. So, graphically it looks like 4K reads are twice as slow as 512 byte reads, but really they are only ~20% slower. And, all SSD drive makers (AFAIK) recommend doing all I/O in blocks the size of a NAND page (4K or 8K). I’d really like to see why there seems to be this 20% performance difference. I look forward to seeing your future posts with results for server-class SSDs.

[1] http://mkp.net/pubs/storage-topology.pdf

]]> By: Slava http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-4983 Slava Fri, 09 Oct 2009 08:15:43 +0000 http://www.rethinkdb.com/blog/?p=223#comment-4983 Sure, here's a graph for block sizes and alignments up to 250KB: http://www.rethinkdb.com/blog/wp-content/uploads/2009/10/graph-blk-alg-25k.png The highest performance still comes from 512B block sizes aligned to 4K boundaries (it gets just a little higher for larger boundaries, but the delta is so small it's uninteresting). After 4K, the optimal alignment is the block size itself, but the number of IOPS never comes close to 512B block sizes. The B-tree block size post did use unaligned reads (well, aligned to 512K). The idea was to show the process we go through, not to provide actual numbers for people to use. The numbers are too dependent on the drive, the workload, and the database to give a single useful result. Sure, here’s a graph for block sizes and alignments up to 250KB:

http://www.rethinkdb.com/blog/wp-content/uploads/2009/10/graph-blk-alg-25k.png

The highest performance still comes from 512B block sizes aligned to 4K boundaries (it gets just a little higher for larger boundaries, but the delta is so small it’s uninteresting). After 4K, the optimal alignment is the block size itself, but the number of IOPS never comes close to 512B block sizes.

The B-tree block size post did use unaligned reads (well, aligned to 512K). The idea was to show the process we go through, not to provide actual numbers for people to use. The numbers are too dependent on the drive, the workload, and the database to give a single useful result.

]]> By: פורומים http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-4893 פורומים Thu, 08 Oct 2009 21:41:11 +0000 http://www.rethinkdb.com/blog/?p=223#comment-4893 I third (?) Brian’s request for larger block sizes. I third (?) Brian’s request for larger block sizes.

]]> By: Colin Percival http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-4843 Colin Percival Thu, 08 Oct 2009 17:20:43 +0000 http://www.rethinkdb.com/blog/?p=223#comment-4843 You noticed that the IOPS numbers from your last post match up exactly against the numbers for *unaligned* reads from this post, right? BTW, I second Brian's request for larger block sizes. You noticed that the IOPS numbers from your last post match up exactly against the numbers for *unaligned* reads from this post, right?

BTW, I second Brian’s request for larger block sizes.

]]> By: Brian Smith http://www.rethinkdb.com/blog/2009/10/page-alignment-on-ssds/comment-page-1/#comment-4838 Brian Smith Thu, 08 Oct 2009 17:03:27 +0000 http://www.rethinkdb.com/blog/?p=223#comment-4838 Why do you only test up to a 4kb block size and stride? It is pretty common for databases even on hard disks to use block sizes of 16kb. Since SSDs normally have to read 128kb at a time for read/erase/write, I'd guess they'd be very good at 128kb aligned reads. Also, is your test utilizing NCQ? I would guess that if you quckly sent 32 4K read requests (on the same aligned 128kb block), you would get a nice boost over sending a 4K read, waiting for a response, sending the next one, etc. Why do you only test up to a 4kb block size and stride? It is pretty common for databases even on hard disks to use block sizes of 16kb. Since SSDs normally have to read 128kb at a time for read/erase/write, I’d guess they’d be very good at 128kb aligned reads.

Also, is your test utilizing NCQ? I would guess that if you quckly sent 32 4K read requests (on the same aligned 128kb block), you would get a nice boost over sending a 4K read, waiting for a response, sending the next one, etc.

]]>