Recently, I got my hands on an old Dell Optiplex 3050 SFF Micro. I plan on setting it up as server for home brewing things like backup, file sharing, NAS and maybe even hosting my own git forge instance. Electricity is pretty expensive in Germany. Therefor, my main focus is to reduce the power draw in idle state, since that’s the one it’s gonna be in most of the time.

Eager to start, I first measured the power consumption of the Optiplex (out of the box with Windows 11 Pro still installed) without a monitor connected and it drew about 18W. Then I measured again with a monitor, mouse & keyboard plugged in and suddenly, the machine only drew around 10W (with occasional random power spikes because Windows). It daunted on me that I had to properly setup Windows and let it run for some time to settle down its background tasks…

Setting up Windows & BIOS#

So, to give this comparison a fair chance, I first had to get all the cumulative updates for Win 11, which took over one (!) hour just downloading & installing new drivers and security patches. After that, I also installed the newest chipset & BIOS updates. The internet recommended to enable ASPM and C-States and additionally I disabled audio ports since they will be virtually never used.

And now, Windows had a surprise up its sleeve that I did not expect: It was drawing as little as 5W, sometimes dipping below, but also regularly above that. I first did not trust my eyes, but I used the same PSU, same outlet, same keyboard & mouse and same Optiplex 3050. The only difference was that I installed some updates and tweaked the bios a little.

I was very intrigued to see what I could achieve with Linux out of the box and maybe even tweaking for power saving…

Setting up Linux#

My favorite flavor of Linux for the past year has been NixOS (I fully committed a couple of months ago and ditched Pop!_OS on my work laptop), so I finally wanted to give it a go for the home lab server world.

Installing vanilla Gnome and and the 6.12 LTS kernel was simple enough. Right out of the gate, I measured 8W, which you know, is already pretty good. Then I realized that the USB drive, with which I installed NixOS drew 1W! So I removed it and was able to get down to about 7W. Now I got excited to see, how great the numbers will look, when I remove the desktop environment. With only a few lines commented (and an SSH port opened), I easily switched from a full desktop environment to headless setup in just a few moments.

However, what I saw I couldn’t believe… Headless Linux was drawing 9W idle. This did not make sense and obviously I was a little bit confused for a moment - enough computer for today.

After a good night’s sleep and I quick web search, I realized that I should probably take a look at the C-states. Thanks to the power of NixOS, I booted back into to the Gnome desktop and installed powertop via the nix-shell - which told me that it spent about 90% in C3. And doing the same thing for the headless configuration, I saw that it spent 98.9% in C2. Clearly, I had done something wrong with my headless configuration.

Power Tweaking#

From scavenging wikis & blogs I gathered that devices connected on the motherboard (i.e. SATA drives, WIFI cards, etc.) can cause the the CPU to never “rest” and enter higher C-states. However, figuring out what exact peripherals cause this behavior is not straight forward.

First I unplugged all my USB peripherals and logged onto the machine only via SSH - without any noticeable effect. Then I started to disable the actual USB ports in the BIOS - again, no use. Also removing the second SATA drive did not make a difference. Slowly, I really hoped it was not the NVME controller that my boot drive was using…

Luckily, I found a pretty useful forum thread which highlighted that certain hardware, despite it being forced in BIOS, doesn’t actually have ASPM enabled in runtime. Checking this can be done with:

$ sudo lspci -vv | grep ASPM

which gave me the following result:

00:1b.0 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #21 (rev f0) (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
00:1c.0 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 (rev f0) (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Disabled; RCB 64 bytes, LnkDisable- CommClk+
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 (rev 01) (prog-if 02 [NVM Express])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller (rev 15)
		LnkCtl:	ASPM L1 Disabled; RCB 64 bytes, LnkDisable- CommClk+

Aha! My ethernet controller actually does not enter ASPM L1 and therefor prevents the CPU from idling further than C2. That also explains why Gnome was able to reach a higher state, since systemd-networkd is handled by the gnome-shell (and probably does some magic).

The forum thread also provided a neat solution for enabling ASPM L1 for my device:

It is possible to force enable aspm for the device by using
echo 1 | sudo tee /sys/bus/pci/drivers/r8169/0000:02:00.0/link/l1_aspm
The corresponding device number eg. “0000:02:00.0” you can get out of lspci

Running this command with the right device address enabled L1 for my ethernet controller and boom - powertop reports 95% C8!

One other thing I noticed was that when I rebooted my system, it did not automatically enter C8. The reason was that I had to rerun sudo powertop -c on every boot (which is little bit annoying, but good enough for now). So I quickly cooked up the following systemd service routine:

systemd.services.optiplex-power-tweaks = {
	description = "Enables ASPM L1 for ethernet controller & runs powertop calibration to reach higher C-states.";
	script = ''
		${pkgs.bash}/bin/bash -c '${pkgs.coreutils}/bin/echo 1 | ${pkgs.sudo}/bin/sudo ${pkgs.coreutils}/bin/tee /sys/bus/pci/drivers/r8169/0000:02:00.0/link/l1_aspm > /dev/null && ${pkgs.sudo}/bin/sudo ${pkgs.powertop}/bin/powertop -c'
		'';
	wantedBy = [ "multi-user.target" ]; # run only at boot
};

The Big Showdown#

Power draw when PC turned off: 0.7W
Windows sleep: 0.9W
Linux sleep: 0.8W

Monitor connected#

Monitor unconnected#

Closing thoughts#

My guess on why Windows was able to to reach such low power draws, albeit with power spikes, is that the desktop environment was able to reach a higher C-state i.e. C6, however was forced into lower ones quite frequently (by the GPU or similar). Which, if you ask me, respect for no manual configuration needed (other than the painful update process). This definitely surprised me the most about this little study.

Vanilla Gnome was very decent since I’d argue that constant power draw of ~7W really comes close to a Raspberry Pi under load. However, the tweaked headless configuration of course takes the cake and swoops the competition under the rug - with constant idle load of 3W, this is some real RPi territory that I actually never expected to come close to!

At last, I am really happy that I looked into C-states and ASPM. This really tells you the story that investigating low level peripheral shenanigans can save you some real money. My little server is now prepared with a very solid base for 24/7 service (with an electricity bill in Germany).