BTRFS – TOO MANY BALANCES – When to do balances
#############################################

We (who is we, btrfs users) shouldn’t do em as much. They dont do much unless the systems allocation is full. Allocation is the TOTAL column.

Because we have to think of what balance does. It just moves around allocations essentially. Allocation is the middle step between having data on the disk and not having data on the disk.

First you have a disk, with empty space —> System decides to write something —> System allocates some data and some metadata space for future write —> System Writes data and metadata

So the write process can be thought of like this:
Allocation -> Write

Also its important that once you allocated data you cant allocated to another type, UNLESS you do a balance.

So the system has to decide whether to allocate chunks towards a certain type of data (data chunk, which is like a block of data, but in btrfs world, its a block or blocks of data of a certain type  – and only that type of data goes there), these chunks are: METADATA (Data about data), SYSTEM (no clue), and DATA (user data).The system has a prediction engine to find out how much METADATA or SYSTEM or DATA chunk allocation, and if it makes a mistake a balance can sort that out.

BALANCE NOT USEFUL
——————-

Here is an example where a balance will not be useful:

# variations of "df"
Filesystem 1024-blocks Used Available Capacity Mounted on
rootfs 4190208 1964332 1876132 52% /
/dev/md127 42923855744 25765476876 17155762420 61% /Volume1

Filesystem Size Used Avail Use% Mounted on
rootfs 4.0G 1.9G 1.8G 52% /
/dev/md127 40T 24T 16T 61% /Volume1

Filesystem 1073741824-blocks Used Available Capacity Mounted on
rootfs 4G 2G 2G 52% /
/dev/md127 40936G 24572G 16362G 61% /Volume1

# btrfs fi show /data
Label: 7c6ebf3a:Volume1 uuid: 736d755a-8651-4cc4-8726-9a783bfc49b7
Total devices 1 FS bytes used 23.95TiB
devid 1 size 39.98TiB used 27.77TiB path /dev/md127

# btrfs fi df /data
Btrfs v0.20-rc1
Data, single: total=27.67TiB, used=23.91TiB
System, DUP: total=8.00MiB, used=2.96MiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=47.00GiB, used=45.76GiB
Metadata, single: total=8.00MiB, used=0.00

 

Notice the total column added up:
27.67 TiB + (8 MiB * 2) + 4 MiB + (47 GiB * 2) + 8 MiB = 27.77 TiB

From “btrfs fi show” the 27.77 is the bigger number on the bottom (so thats an easy way to find that number instead of doing that calculation on paper/head). Allocation should always be equal to or bigger then the used sizes. Because Allocation happens first before space is used.

Its not useful because we still have 39.98 – 27.88 = 12 TiB left for allocation. Plenty of room left for allocation

So a balance here will not be useful.

BALANCE USEFUL
—————

# df -ha
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 75G 60G 852M 99% /

# btrfs fi df /
Data: total=50.00GB, used=49.17GB (the difference here is 849 MB)
System: total=32.00MB, used=4.00KB
Metadata: total=24.50GB, used=9.86GB

So here we have an unbalanced case.
Notice that the ALLOCATION on the unit (again the total table summed up):
50 GiB + 32 MiB + 24.50 GiB = 74.50 (which is VERY close to 75 GB)

Notice that the size of sda1, the volume where btrfs is, is also 75 GiB (From the DF output).

Now that we know its prettyful. Lets see what we have for balance (how much free space in each available chunk type):

Data has 50 GB allocated and used 49.17 GB, so thats less than 1 GB free: DATA freespace 1 GB
System has 32 MB allocated and used 4 KB, so thats less than 32 MB Free: SYSTEM freespace 32 GB
Metadata has 24.50 GB allocated and used 9.86 GB, so thats about 14 GB free: METADATA freespace 14 GB

Now here is a thought, why should METADATA have more freespace? Shouldnt data have more. Data in general takes up more then metadata. Well the answer is, it shouldnt, this was just bad allocation on the btrfs allocator part. So we can fix this with the BTRFS BALANCE command. Without doing the balance, we simply cant write data here, the system is pretty much performing as full even though if you did the math there is about 15 GB of free space left.

How to balance:

btrfs balance start -dusage=5 start /

REQUIREMENTS FOR BALANCE: the data needs to be mounted RW (read write) and also it will take some time. You will be able to use the data in the mean time (it will just take a performance hit)

My favorite way to balance:
Start at 0 then move up to 50 (final number can vary depending on the TBs of data involved, the more TB, the longer this operation will take). The step interval doesnt really matter, i usually start off small jumps then bigger jumps.

for i in 0 2 5 10 20 30 50 75; do echo "====Balancing with dusage $i===="; btrfs balance start -dusage=$i; done;

By the way I found that its, a little bit quicker (or maybe its just quicker in my imagination) to do it in steps then single handedly in one big operation that does it like so: btrfs balance start -dusage=75; done; 

Note: dusage=0 says that it will only move chunks that are 0% full (meaning they are empty chunks), so empty chunks will get moved around for better placement. dusage=5 means that it will take chunks that are 5% full or below (mostly empty) and move those around.