ファイルサーバーに使うブートディスクはUSBメモリでも良いのでは感

大学４年生になったら履修する科目は無かったはずなんですが
今年も２年生に混じって微積IIを履修することになりました。

 

動機

今までファイルサーバーには8GB SSDを２つ載せてRAID1で組んでいたのですが
ケース内を圧迫され、HDDを積めなくなるのは嫌だったのでUSBメモリで運用してみました。

 

使用するUSBメモリ

SanDisk Extreme USB3.0 フラッシュメモリー 32GB
型番：SDCZ80-032G-J57

ファイルサーバーとだけで使用するなら8GBでも十分なんですが
PT2/PT3とかMirakurunとか載せる場合は32GBぐらいないとキツイかもしれません。

 

CentOS 7.3.1611をUSBにインストールしてみた

[root@file /]# df -h .
ファイルシス   サイズ  使用  残り 使用% マウント位置
/dev/sdb4         26G  3.8G   22G   15% /

インストール直後はちょっと言い過ぎたんですが
インストール直後にyum updateした後は1.8GBぐらい使用していました。

余裕を持って32GBぐらいにしておくと良いかもしれない。

 

fioでベンチマーク

USBメモリなので関係ないのかな？って思うんですが
non-bufferredです。

Seq Read

bs -> 4k, jobs -> 64

[root@file tmp]# fio -filename=/tmp/test2g -direct=1 -rw=read -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
fio-2.2.8
Starting 64 processes
file1: Laying out IO file(s) (1 file(s) / 2048MB)
Jobs: 64 (f=64): [R(64)] [100.0% done] [62280KB/0KB/0KB /s] [15.6K/0/0 iops] [eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=2257: Sun Apr 23 17:51:33 2017
read : io=607596KB, bw=60693KB/s, iops=15173, runt= 10011msec
clat (usec): min=639, max=19736, avg=4212.60, stdev=1157.23
lat (usec): min=640, max=19736, avg=4212.75, stdev=1157.24
clat percentiles (usec):
|  1.00th=[ 1640],  5.00th=[ 2352], 10.00th=[ 3088], 20.00th=[ 3696],
| 30.00th=[ 3888], 40.00th=[ 4048], 50.00th=[ 4192], 60.00th=[ 4256],
| 70.00th=[ 4448], 80.00th=[ 4640], 90.00th=[ 5088], 95.00th=[ 5920],
| 99.00th=[ 8768], 99.50th=[ 9920], 99.90th=[12864], 99.95th=[14400],
| 99.99th=[17536]
bw (KB  /s): min=  637, max= 1048, per=1.56%, avg=947.16, stdev=74.01
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=2.94%, 4=34.46%, 10=62.12%, 20=0.47%
cpu          : usr=0.06%, sys=0.12%, ctx=152072, majf=0, minf=2309
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=151899/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=607596KB, aggrb=60692KB/s, minb=60692KB/s, maxb=60692KB/s, mint=10011msec, maxt=10011msec
Disk stats (read/write):
sdb: ios=18051/4, merge=110475/1, ticks=90614/124, in_queue=91866, util=98.84%

IOPS: 15173

 

Seq Write

bs -> 4k, jobs -> 64

[root@file tmp]# fio -filename=/tmp/test2g -direct=1 -rw=write -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
fio-2.2.8
Starting 64 processes
Jobs: 64 (f=64): [W(64)] [100.0% done] [0KB/50596KB/0KB /s] [0/12.7K/0 iops] [eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=2330: Sun Apr 23 17:54:35 2017
write: io=477124KB, bw=47660KB/s, iops=11914, runt= 10011msec
clat (usec): min=170, max=32756, avg=5365.53, stdev=2814.40
lat (usec): min=170, max=32756, avg=5365.78, stdev=2814.40
clat percentiles (usec):
|  1.00th=[ 1224],  5.00th=[ 2224], 10.00th=[ 2864], 20.00th=[ 3568],
| 30.00th=[ 4192], 40.00th=[ 4576], 50.00th=[ 4960], 60.00th=[ 5408],
| 70.00th=[ 5792], 80.00th=[ 6432], 90.00th=[ 7520], 95.00th=[ 9536],
| 99.00th=[19072], 99.50th=[20096], 99.90th=[23936], 99.95th=[25984],
| 99.99th=[28800]
bw (KB  /s): min=  646, max=  948, per=1.56%, avg=744.07, stdev=53.19
lat (usec) : 250=0.01%, 500=0.02%, 750=0.20%, 1000=0.38%
lat (msec) : 2=2.57%, 4=22.65%, 10=69.74%, 20=3.85%, 50=0.59%
cpu          : usr=0.05%, sys=0.11%, ctx=119664, majf=0, minf=2165
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=119281/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=477124KB, aggrb=47659KB/s, minb=47659KB/s, maxb=47659KB/s, mint=10011msec, maxt=10011msec
Disk stats (read/write):
sdb: ios=0/14091, merge=0/88140, ticks=0/102728, in_queue=103779, util=98.86

IOPS: 11914

 

Rand Read

bs -> 4k, jobs -> 64

[root@file /]# fio -filename=/tmp/test2g -direct=1 -rw=randread -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
fio-2.2.8
Starting 64 processes
Jobs: 64 (f=64): [r(64)] [100.0% done] [12264KB/0KB/0KB /s] [3066/0/0 iops] [eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=2441: Sun Apr 23 18:16:48 2017
read : io=122620KB, bw=12231KB/s, iops=3057, runt= 10025msec
clat (msec): min=1, max=223, avg=20.89, stdev=14.73
lat (msec): min=1, max=223, avg=20.89, stdev=14.73
clat percentiles (usec):
|  1.00th=[ 1464],  5.00th=[ 2960], 10.00th=[ 4832], 20.00th=[ 8640],
| 30.00th=[12352], 40.00th=[16064], 50.00th=[19840], 60.00th=[23424],
| 70.00th=[27264], 80.00th=[31104], 90.00th=[35584], 95.00th=[38656],
| 99.00th=[76288], 99.50th=[99840], 99.90th=[146432], 99.95th=[162816],
| 99.99th=[187392]
bw (KB  /s): min=   71, max=  279, per=1.56%, avg=191.18, stdev=27.39
lat (msec) : 2=2.39%, 4=5.33%, 10=15.99%, 20=26.99%, 50=47.43%
lat (msec) : 100=1.38%, 250=0.50%
cpu          : usr=0.02%, sys=0.03%, ctx=30762, majf=0, minf=2324
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=30655/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=122620KB, aggrb=12231KB/s, minb=12231KB/s, maxb=12231KB/s, mint=10025msec, maxt=10025msec
Disk stats (read/write):
sdb: ios=30227/0, merge=8/0, ticks=630666/0, in_queue=631697, util=98.86%

IOPS: 3057

 

Rand Write

bs -> 4k, jobs -> 64

[root@file /]# fio -filename=/tmp/test2g -direct=1 -rw=randwrite -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
fio-2.2.8
Starting 64 processes
Jobs: 64 (f=64): [w(64)] [100.0% done] [0KB/6340KB/0KB /s] [0/1585/0 iops] [eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=2512: Sun Apr 23 18:20:09 2017
write: io=64416KB, bw=6416.6KB/s, iops=1604, runt= 10039msec
clat (usec): min=194, max=354223, avg=39799.34, stdev=28106.61
lat (usec): min=195, max=354223, avg=39799.67, stdev=28106.61
clat percentiles (msec):
|  1.00th=[    3],  5.00th=[    6], 10.00th=[   10], 20.00th=[   17],
| 30.00th=[   24], 40.00th=[   31], 50.00th=[   38], 60.00th=[   45],
| 70.00th=[   52], 80.00th=[   60], 90.00th=[   69], 95.00th=[   76],
| 99.00th=[  133], 99.50th=[  194], 99.90th=[  273], 99.95th=[  297],
| 99.99th=[  347]
bw (KB  /s): min=   39, max=  174, per=1.56%, avg=100.31, stdev=18.73
lat (usec) : 250=0.01%, 500=0.02%, 750=0.01%, 1000=0.03%
lat (msec) : 2=0.35%, 4=2.71%, 10=8.10%, 20=14.41%, 50=41.43%
lat (msec) : 100=31.33%, 250=1.45%, 500=0.15%
cpu          : usr=0.01%, sys=0.03%, ctx=16174, majf=0, minf=2174
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=16104/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=64416KB, aggrb=6416KB/s, minb=6416KB/s, maxb=6416KB/s, mint=10039msec, maxt=10039msec
Disk stats (read/write):
sdb: ios=0/15865, merge=0/2, ticks=0/631218, in_queue=632723, util=98.87%

IOPS: 1604

 

まとめ

種類	IOPS	スループット(KB/s)	スループット(MB/s)	レイテンシ
Seq Read	15173	60693	60.7	10msec
Seq Write	11914	47660	47.7	10msec
Rand Read	3057	12231	12.2	50msec
Rand Write	1604	6416.6	6.4	50msec

ファイルサーバーでしか使わないなら必要十分だと思います。

心配性の人はUSBを数本用意してRAIDで運用したら楽しいかもしれません?