SLABTOP BTRFS DURING MOUNTS
#############################

Its intersting to watch the mounts with slabtop and top. Btrfs moutns can take a minute because of the metadata work they do during the mount.

This script below is interesting to run on a box right before you commence a mount of a btrfs system it shows the output of slabtop (kernel modules ram use), specifically just the ones that mention btrfs, also its showing the resource use via top (showing load average, cpu utilization, ram use, and the top processes in use in the device)

while true;
do
clear
echo "=====`date`====="
(slabtop -o | egrep -i "active|Average|btrfs";
echo "----";
top -cbn1 | head -n 15 | grep .;)
sleep 1
done

SUMMARY: data is mounting if btrfs_extent_buffer is growing, once its mounted it will stabilize at a number, then when you write or read that btrfs_Extent_buffer will grow. in the mean time btrfs_free_space will stay at about the same size and btrfs_extent_map will as well (unless of course alot of writes happen to where those need to change accordingly). Remember these numbers represent the amount of metadata of BTRFS in the RAM (AKA: btrfs kernel module’s slab information – every module/driver has something in slabtop near about – I just grepped out the btrfs stuff)

NOTE: the only reason that btrfs_free_space exists even after the unmount is because the root here is also btrfs. (its only 4 gigs big, so before data, md127, is mounted btrfs_free_space is only 156K and then it grows to 406K through out the mount of data)

THE DATA IM MOUNTING (this is on 1 Raid device that is composed of 7 drives and 1 spare):

# cat /proc/mdstat
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md127 : active raid6 sde3[0] sdd3[7](S) sdc3[6] sda3[5] sdb3[4] sdi3[3] sdh3[2] sdf3[1]
9743313920 blocks super 1.2 level 6, 64k chunk, algorithm 2 [7/7] [UUUUUUU]

 

# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 4190208 558628 3218540 15% /
tmpfs 10240 0 10240 0% /dev
/dev/md0 4190208 558628 3218540 15% /
tmpfs 10240 0 10240 0% /dev
/dev/md127 9743313920 6741374960 2921121192 70% /data

 

# btrfs fi df /data
Data, single: total=6.77TiB, used=6.25TiB
System, DUP: total=8.00MiB, used=752.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=52.00GiB, used=13.48GiB
Metadata, single: total=8.00MiB, used=0.00

OUTPUT BELOW EXCLUDES OUTPUT OF “top” JUST HAS THE “slabtop” INFORMATION

BEFORE MOUNT:

=====Mon Apr 14 14:27:33 EDT 2014=====
Active / Total Objects (% used) : 74055 / 75624 (97.9%)
Active / Total Slabs (% used) : 2192 / 2192 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 26727.49K / 27306.01K (97.9%)
Minimum / Average / Maximum Object : 0.01K / 0.36K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
2496 2373 95% 0.06K 39 64 156K btrfs_free_space <=== exists because md0 is btrfs

NOTE: I only have btrfs_free_space because my root is btrfs (if my root was ext this wouldnt be here and it would be empty because there wouldnt be any btrfs modules to run)

NOTE: in arm units you will not have btrfs slabs when your in sysroot (unless you have md127 mounted)
DURING MOUNT (waiting on mount to complete):

=====Mon Apr 14 14:27:56 EDT 2014=====
Active / Total Objects (% used) : 90102 / 90948 (99.1%)
Active / Total Slabs (% used) : 2619 / 2619 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 29025.96K / 29253.97K (99.2%)
Minimum / Average / Maximum Object : 0.01K / 0.32K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
7124 7124 100% 0.15K 274 26 1096K btrfs_extent_map
2496 2365 94% 0.06K 39 64 156K btrfs_free_space
1680 1680 100% 0.33K 70 24 560K btrfs_extent_buffer <=== growing

NOTE: the second I mount the data volume, we get 2 more enteries for btrfs in slabtop and specifically we see btrfs_extent_buffer grow. Meanwhile the mount command is just sitting there acting like its stuck! (its not as you can see the btrfs_extent_buffer is growing)

 STILL DURING MOUNT (waiting on mount to complete):

=====Mon Apr 14 14:28:04 EDT 2014=====
Active / Total Objects (% used) : 100065 / 100911 (99.2%)
Active / Total Slabs (% used) : 3011 / 3011 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 32498.02K / 32746.09K (99.2%)
Minimum / Average / Maximum Object : 0.01K / 0.32K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
7124 7124 100% 0.15K 274 26 1096K btrfs_extent_map
6312 6312 100% 0.33K 263 24 2104K btrfs_extent_buffer <=== growing
2496 2365 94% 0.06K 39 64 156K btrfs_free_space

NOTE: btrfs_extent_buffer is growing and making its way to the top of slabtop

AFTER MOUNT:

=====Mon Apr 14 14:29:08 EDT 2014=====
Active / Total Objects (% used) : 134607 / 135561 (99.3%)
Active / Total Slabs (% used) : 4374 / 4374 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 43762.45K / 44022.17K (99.4%)
Minimum / Average / Maximum Object : 0.01K / 0.32K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
21624 21624 100% 0.33K 901 24 7208K btrfs_extent_buffer
7124 7124 100% 0.15K 274 26 1096K btrfs_extent_map
2496 2432 97% 0.06K 39 64 156K btrfs_free_space

NOTE: the mount is complete, now the btrfs_extent_buffer has stabilized at 901 objects at 7208K (7 megs of RAM). You can cruise around the FS as I did and it will not grow or shrink (who knows maybe I needed to go deeper in)

DOING A READ HEAVY OPERATION:

=====Mon Apr 14 14:31:25 EDT 2014=====
Active / Total Objects (% used) : 151616 / 152706 (99.3%)
Active / Total Slabs (% used) : 5033 / 5033 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 50198.69K / 50474.62K (99.5%)
Minimum / Average / Maximum Object : 0.01K / 0.33K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
26352 26352 100% 0.33K 1098 24 8784K btrfs_extent_buffer
11596 11596 100% 0.15K 446 26 1784K btrfs_extent_map
2496 2432 97% 0.06K 39 64 156K btrfs_free_space

NOTE: doing a deep read of everything on the FS, filesystem, made the btrfs_extent_buffer grow even more

READ EVERYTHING: find /data -exec echo {} \; -exec cp {} /dev/null \;

DOING WRITE HEAVY OPERATION:

=====Mon Apr 14 14:32:31 EDT 2014=====
Active / Total Objects (% used) : 196192 / 197438 (99.4%)
Active / Total Slabs (% used) : 6770 / 6770 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 66242.14K / 66543.10K (99.5%)
Minimum / Average / Maximum Object : 0.01K / 0.34K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
53112 53112 100% 0.33K 2213 24 17704K btrfs_extent_buffer
16198 16198 100% 0.15K 623 26 2492K btrfs_extent_map
4544 4544 100% 0.06K 71 64 284K btrfs_free_space

NOTE: a write operation is just as heavy and the btrfs_extent_buffer kept on growing (notice the btrfs_free_space map is nice and small considering we have ALOT of data)

NOTE: this is really just heavy on both read and write 50/50

WRITE HEAVY COMMAND:

find /data -exec echo {} \; -exec cp {} /data/deleteme \;

DOING NOTHING (RIGHT BEFORE UNMOUNT):

=====Mon Apr 14 14:34:09 EDT 2014=====
Active / Total Objects (% used) : 242945 / 244250 (99.5%)
Active / Total Slabs (% used) : 8590 / 8590 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 84346.53K / 84734.30K (99.5%)
Minimum / Average / Maximum Object : 0.01K / 0.35K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
83880 83880 100% 0.33K 3495 24 27960K btrfs_extent_buffer
16198 16198 100% 0.15K 623 26 2492K btrfs_extent_map
6656 6656 100% 0.06K 104 64 416K btrfs_free_space

NOTE: Right before the unmount of the filesystem that has the data, we did heavy read and write operations for a few seconds each.

1 SECOND INTO UNMOUNT: 

=====Mon Apr 14 14:35:04 EDT 2014=====
Active / Total Objects (% used) : 263351 / 265016 (99.4%)
Active / Total Slabs (% used) : 9419 / 9419 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 93849.84K / 94535.25K (99.3%)
Minimum / Average / Maximum Object : 0.01K / 0.36K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
106392 106392 100% 0.33K 4433 24 35464K btrfs_extent_buffer
7744 7744 100% 0.06K 121 64 484K btrfs_free_space
7254 7190 99% 0.15K 279 26 1116K btrfs_extent_map

NOTE: during the unmount the btrfs_extent_buffer grows and then bam disappears

UNMOUNT:

=====Mon Apr 14 14:35:11 EDT 2014=====
Active / Total Objects (% used) : 74424 / 76690 (97.0%)
Active / Total Slabs (% used) : 2242 / 2242 (100.0%)
Active / Total Caches (% used) : 68 / 119 (57.1%)
Active / Total Size (% used) : 26848.49K / 27856.91K (96.4%)
Minimum / Average / Maximum Object : 0.01K / 0.36K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
2496 2373 95% 0.06K 39 64 156K btrfs_free_space

NOTE: btrfs_extent_buffer disappeared and we are back to where we started more or les