Continue wrigin power monitoring post

content/post/custom-power-monitoring.md

@@ -11,9 +11,11 @@ draft = true
 
 +++
 
-So you've got a homelab. It runs great, you've got dozens of VMs, and a spiffy storage cluster, and the hardware wasn't cheap. But, unless you live in Québec, you also probably care about another big cost sink: power usage. Here in Ontario I'm paying anywhere from 8¢ to 18¢ per kilowatt-hour (kWh) of energy, and with my rack totaling over 1600W normally, the power usage is substantial. The problem, however, is determining how much each device is using. Sure, the UPS can give me an overall power usage figure, but with 8 servers, 2 routers, 2 switches, and several other devices, knowing exactly how much each machine is using helps enable even more granular control of the power usage.
+So you've got a homelab. It runs great, you've got dozens of VMs, and a spiffy storage cluster, and the hardware wasn't cheap. But, unless you live in Québec, you probably also care about another big cost sink: power usage. Here in Ontario I'm paying anywhere from 8¢ to 18¢ per kilowatt-hour (kWh) of energy, and with my rack totaling over 1600W normally, the power usage is substantial. The problem, however, is determining how much each device is using. Sure, the UPS can give me an overall power usage figure, and some (newer) devices can report their own power usage, but with 8 servers, 2 routers, 2 switches, and several other devices of significant age, knowing exactly how much each machine is using helps enable even more granular control and monitoring of the power usage.
 
-The most obvious solution to this problem is what is usually referred to as a "Metered Per-Port PDU" in marketing-speak. Simply-put, it's a large power bar (Power Distribution Unit, for the acronym-unfriendly) that contains a network port and a series of current monitors on each port, allowing you to collect information about per-port power usage via SNMP or a WebUI. Most of the big PDU vendors make one of these devices (though with different names - marketing-speak, after all), however they are not cheap. The lowest cost on I could find was from Tripp-Lite at $1200 CAD new. That's a pretty penny to pay for power distribution, so instead, in my always-present DIY mindset, I started thinking about building my own.
+The most obvious solution to this problem is what is usually referred to as a "Metered Per-Port PDU" in marketing-speak. Simply-put, it's a large power bar (a Power Distribution Unit or PDU) that contains a network port and a series of current monitors on each port, allowing you to collect information about per-port power usage via SNMP or a WebUI. Most of the big PDU vendors make one of these devices, though with different names - marketing-speak, after all. However they are not cheap. The lowest cost one I could find that matched my common homelab specs of 120V 20A was from Tripp-Lite at $1200 CAD new. That's a pretty penny to pay for power distribution! So instead, in my always-present DIY mindset, I started thinking about building my own.
+
+# Part 1 - The Theory
 
 ## The basics of power usage and monitoring
 
@@ -27,22 +29,183 @@ P = V * I
 
 A/C power complicates this ideal formula slightly. A/C power works on a fluctuating voltage in a sine wave, which alternates at a specific frequency. In North America, this is 60Hz (cycles per second). What this means it that every 1/60th of a second, the voltage peaks (120V for a standard household circuit as mentioned), and then dips down to 0V before repeating. And as you know from the formula above, when the voltage is 0, the power usage will also be 0, so the effective power usage also looks like a sine wave with a frequency of 60Hz. While you usually don't notice this fluctuation in your daily life, when trying to measure the power of a circuit, it becomes very important to take this into account. The next section goes over how to create a current monitoring circuit for A/C power.
 
-The last thing to touch on for those unfamilliar is the idea of a Kilowatt-Hour, or kWh. One kWh is equal to 1000W (1 kW) of power used continuously for 1 hour. Since this is a time-based measurement, versus the instantaneous measurement of the `P = V * I` formula, it is the most common unit of billing for electric power. As a concrete example, assumming the previous example load of 1A at 120V, working out to 120W, you simply multiply this out by 1 hour to get the usage of 0.120 kWh of energy usage every hour. Over two hours it would use 0.240 kWh of power, and etcetera for as long as the device is active. When you get your monthly power bill, this is how the total power usage is calculated.
+The sine wave power mentioned above also adds another complication: the idea of real power. While the [details are complex](https://en.wikipedia.org/wiki/AC_power#Active.2C_reactive.2C_and_apparent_power), if you're just trying to figure our "how much power am I using", this isn't a huge deal: you can just know you're always using "less" than what you calculate with the simple formula. But devices like UPSes get complicated. Most are rated in Volt-Amps, such as my APC Smart-UPS 2200. It's rated at 2200 VA, but in reality it can't actually power 2200 _Watts_ - the VA value represents the _apparent power_ maximum of the UPS. The real supported Watt value is actually lower, around 1800W for this model. What this means, is, you may see yourself using >1800 "Watts" by simple calculation, and you might still be "fine", because your _real power_ isn't exceeding 1800W. But if your usage spikes, you can very quickly learn that what you thought was "enough", wasn't, even though your "Watts" value is still under 2200. Always size based on the "real" Watts value rather than the rated VA value of a UPS, or alternatively factor in the power factor of your devices into your calculations. For simplicity though, for the rest of this post I'll use the basic definition of "Watt" when describing caltulations and values.
+
+The last thing to touch on for those unfamilliar is the idea of a Kilowatt-Hour, or kWh. One kWh is equal to 1000W (1 kW) of power used continuously for 1 hour. Since this is a time-based measurement, versus the instantaneous measurement of the `P = V * I` formula, it is the most common unit of billing for electric power. As a concrete example, assumming the previous example load of 1A at 120V, working out to 120W, you simply multiply this out by 1 hour to get the usage of 0.120 kWh of energy every hour. Over two hours it would use 0.240 kWh of power, and etcetera for as long as the device is active. When you get your monthly power bill, this is how the total power usage is calculated.
 
 ## An A/C monitoring circuit
 
-With the theoretical underpinnings of this project out of the way, we can begin looking at devices to measure the current of an A/C circuit. Given the formula above, as long as the voltage is known, the only thing we need to measure is the current in Amperes, and we'll be able to do some quick math to get the wattage value. Luckily, there's a very easy way to measure the current of a circuit. Called the 'Hall effect', it relies on the magnetic field of a powered conductor, and while the effect is far to complicated to explain here ([Wikipedia is a good source](https://en.wikipedia.org/wiki/Hall_effect)), suffice to say that it gives a very accurate measurement of the current through a conductor at any given time. The Hall effect works equally well with both A/C and D/C power.
+With the theoretical underpinnings of this project out of the way, we can begin looking at devices to measure the current and voltage of an A/C circuit. This project will use two kinds of sensors: current, and voltage. Current sensors will be installed in-line with every socket, while a single voltage sensor will be installed on each circuit. Given the formula above, we'll be able to do some quick math to get the wattage value from these numbers as well.
+
+The voltage sensors are the simplest - they feature a ~120V transformer which drops the voltage to a more reasonable number, then an OPAMP circuit to filter, regulate, and process that voltage to give a reading. The ones I use in this project are unbranded as far as I can tell, but can be found on eBay as a "Single-phase Voltage Sensor" under `Business & Industrial > Automation, Motors & Drives > Sensors > Other Sensors`. They also have an adjustable output sensitivity that must be calibrated using a reference voltmeter, which we get to later on during the build. With one sensor per circuit we can know the voltage of any outlet in that circuit.
+
+The current sensors are more complicated. There's a very easy way to measure the current of a circuit, called the 'Hall effect', which relies on the magnetic field of a powered conductor to generate a reference voltage which can then be mathematically transformed into an amperage value. While the effect is far to complicated to explain here ([Wikipedia is a good source](https://en.wikipedia.org/wiki/Hall_effect)), suffice to say that it gives a very accurate measurement of the current through a conductor at any given time. The Hall effect works equally well with both A/C and D/C power.
 
 Hall effect sensors come in two main types. The first is a wound core through which the hot wire of the circuit is placed. This is a "non-invasive" Hall sensor, and while it works quite well, it is a lot bulkier than the second type. This second type is an in-line Hall sensor, best exemplified by the ACS712-T sensor based on the Allegro chip. This simple 8-pin chip takes the entire current through four of its pins, and via an internal coil and detector, outputs a variable D/C voltage based on the sensed current. To save space as well as cost, since I'll be building the entire PDU from scratch anyways, I went with these sensor chips instead of the "non-invasive" type.
 
 The ACS712-T comes in three different varieties, each rated for a different current load: 5A, 20A, and 30A. Each variety outputs a slightly different D/C voltage per Ampere of through current, which allows for varying sensitivity up to 30A. For my purposes, the 20A model was both the lowest I could go (5A would not be enough for the larger servers), while the 30A would be overkill and its extremely small D/C voltage variation would have made "long"-distance measuring of the voltage more problematic.
 
+Now that we know about the two kinds of sensors we'll need, we can proceed to the parts list.
 
-## Building the PDU
+# Part 2 - The Parts List
 
-## Building the monitoring circuits
+All the following parts can be found online (sensors, Arduino) or at (Canadian) Home Depo (electrical supplies, tools). No part should be optional for this build to ensure safety and code compliance.
 
-## The software side
+### The power monitoring components
 
-## Conclusion
+2x "Single-phase Voltage Sensor" modules (one per circuit)
+20x "ACS712-T Current Sensor" modules (one per outlet socket)
+1x Arduino Uno
+1x Arduino MUX shield (Mayhew Labs) (provides up to 48 analog inputs, enough for all our sensors)
+1x Raspberry Pi (or other Linux-capable system)
+10+ft Cat5e cable
+
+### The A/C electrical components
+
+1x 2"x4"x4' mounting board
+10x NEMA-15 dual-socket outlets (Decora-style)
+5x 3-gang plastic stud-mount electrical boxes
+3x 2-gang plastic stud-mount electrical boxes
+20+ft 120V 15-3 ROMEX wiring
+2x Socketless GFCIs
+2x Switch + Pilot light units
+5x 3-gang Decora faceplates (for outlet boxes)
+5x Decora single-outlet blanker (for covering current sensors)
+1x 2-gang Decora faceplate (for GFCI box)
+1x 2-gang traditional socket faceplate (for switch box)
+1x Blank faceplate (for voltage sensor box)
+2x lengths of rubber cord for power leads
+2x 15-20A plugs for power leads
+Wire nuts/Marrettes (Yellow size 4x14ga)
+Electrical tape
+Small wood screws (for securing the boxes to the board)
+
+### Tools
+
+Wire strippers (good, 24ga to 12 ga)
+Side cutters
+Needle-nose pliers (for manipulating wires)
+Pliers (for twisting wires)
+Screwdrivers (Phillips 2+ and Flat)
+
+# Part 3 - The Build
+
+Now it's time to build your own PDU! This is written as a step-by-step guide on how to complete the PDU, going off my experience and featuring pictures of some completed parts during construction. Note that the voltage sensors were added after the main build was complete and are not shown until their step, along with the proper 2x4.
+
+## Part 3.1 - Installing the boxes
+
+1. Begin with the 2"x4", and attach the boxes vertically along the short length of the board using the small wood screws. The box order should be, from top to bottom: Voltage (2-gang), 5x Outlet (3-gang), GFCI (2-gang), Switch (2-gang).
+
+![Empty boxes attached to board](/images/pdu/boxes.png)
+
+## Part 3.2 - Preparing the wires
+
+1. Cut roughly 12-inch pieces of 15-3 wire, one for each point that a circuit traverses two boxes, and several pieces for connecting wires. 
+
+2. Strip the outer coating except for at least 4 inches - this leaves coated wire to bridge the boxes. Your measurement might vary.
+
+3. Insert each piece of wire into its boxes. This step is quite time-consuming; I recommend pushing the outer coating to one end, feedind that end out one box and into the other, and using screwdrivers to prop open the locking clips. Once fed, the inner wires can be pushed or pulled through further using neetle-nose pliers.
+
+![Wiring between boxes](/images/pdu/box-outer-wiring.png)
+
+4. Cut the connecting pieces of wire. These should be between 8cm and 12cm long to lead to the power sensors and sockets. Measure carefully to avoid having too-short wires later! For each outlet, you will need 4 hot leads (2 from trunk to sensor, 2 from sensor to outlet), 1 neutral lead, and 1 ground lead. Set aside the sensor-to-outlet wires for _Part 3.4_.
+
+5. Begin assembling the box wiring. For the ground and neutral connections, strip and twist together the lead wires from each side of the trunk, and the wire to the socket, then cover with a marrette. For the hot connections, strip and twist together the lead wires from each side of the trunk, and the two wires to the sensors, then cover with a marrette. Push the marretted bundles down towards the bottom of the box. Use the needle-nose pliers to tweak the shape and position of each lead wire to accomodate the sensors and sockets later, and strip off 1cm of insulation from the end of each sensor lead wire.
+
+![Lead wiring](/images/pdu/box-inner-wiring.png)
+
+6. Install the GFCI outlets, since these are static. Consult the GFCI documentation for details on the wiring.
+
+![Installed GFCIs](/images/pdu/gfcis.png)
+
+
+## Part 3.3 - Assembling the sensor modules
+
+1. Recycle three pieces, each approximately 2 inches in lenght, of ROMEX outer coating to act as insulating buffers between the sensors.
+
+2. Attach a sensor unit to one side of each insulating piece using Cyanoacrylate glue, and let dry.
+
+3. Glue each sensor module to the next, and the fourth module to the outside of the last piece, producing a chain of isolated sensors, and let dry.
+
+![A sensor module](/images/pdu/sensor-module.png)
+
+## Part 3.4 - Assembling the outlets
+
+1. Break the hot bridge between the two sockets on the outlet using needle-nose pliers. This isolates the two hot screws and therefore the two sockets from each other, allowing each to be monitored separately.
+
+2. Attach a piece of hot lead wire to each hot screw of the outlet. Strip 1cm of insulation off the far side, and fold under the socket.
+
+Note that the lead lengths must vary for the sensor-near and sensor-far outlets.
+
+![Outlet-side wiring](/images/pdu/outlet-wiring.png)
+
+## Part 3.5 - Installing the outlets
+
+1. Begin installing each socket into its box, ensuring the sensor leads are clear of the marretted bundles in each box. The hot leads from the trunk to the sensors should each be on the right of the leads from the switch when the sensor gang is closest to you; use needle-nose pliers to position them correctly before proceeding.
+
+2. Attach the neutral and ground leads to the outlet.
+
+3. Screw in the outlet, allowing the hot and neutral leads to compress into the box and ensuring the hot leads remain in position.
+
+![Installed outlet wiring](/images/pdu/ready-sensor-leads.png)
+
+## Part 3.6 - Wiring up the sensors
+
+1. Cut five lengths (one per outlet box) of CAT5e cable to act as sensor leads. Conveniently, with 4 pairs, we have four sensor data wires, two power wires, and a spare pair should one of the others fail, so only a single cable per box is needed. Each length should be shorter, leading to all 5 ending at the same height out the top box.
+
+2. Feed the cables through the boxes on the sensor side, leaving plenty of slack. Bundle the upper ends together for later.
+
+![Data cables in the boxes](/images/pdu/data-cables-box.png)
+
+![Data cables bundled](/images/pdu/data-cables-bundled.png)
+
+3. Install the sensor modules. Use the needle-nose pliers to ensure everything is aligning correctly, and trim wires as necessary. I found inserting the modules at 45-degrees then gently bending the wires back down was easiest.
+
+![Single installed sensor module](/images/pdu/installed-sensor-module.png)
+
+![Several installed sensor modules](/images/pdu/installed-sensor-modules.png)
+
+4. Wire up female leads to the sensor cables. Connect the + and - power wires to their respective bundles to distribute power to each module.
+
+![Power leads](/images/pdu/sensor-power-leads.png)
+
+5. Wire up a lead to each sensor wire, taking careful note of and/or standardizing on your colours; this makes figuring out which lead is which sensor much easier later. Wrap the connections in electrical tape for safety and rigity, then connect the leads to each sensor.
+
+![Attached leads](/images/pdu/sensor-attached-leads.png)
+
+6. Cover the sensor gang with a blanker and install the faceplate.
+
+![Blanker installed](/images/pdu/blanker-installed.png)
+
+![Faceplate installed](/images/pdu/faceplate-installed.png)
+
+## Part 3.7 - The lead cables
+
+This part has no pictures because I forgot to take them!
+
+1. In the bottom box, wire up the switch and pilot light combo to each circuit. Feed in each length of rubber wiring, and connect it to its respective circuit's switch. Bind together the neutral connections from each pilot light and the two cables, and pass the hot line through the switch. Cover the box.
+
+2. Wire up the plug to each length of wire to form the source of each circuit.
+
+## Part 3.8 - The voltage sensors
+
+After the fact, having built the PDU with support for current readings only, I regretted not adding voltage sensors as well, to provide the full suite of power monitoring and a true wattage value like I set out to do.
+
+1. Position the incoming wires to allow the module to fit vertically in the box. Strip off 1cm of insulation from each wire.
+
+y. Install each circuit's voltage sensor between the hot and neutral leads, being careful to keep them as far apart as the sensor allows.
+
+3. Connect up the voltage sensor leads, similar to the current leads. To exit the box I cut a notch as these wires were short, but they can be fed more cleanly out the back of the box using longer wire lengths. Both ends should be female to connect to the Mux shield pins.
+
+## Part 3.9 - The Mux shield
+
+# Part 4 - The Software
+
+## Part 4.1 - The Arduino
+
+## Part 4.2 - The Raspberry Pi
+
+## Part 4.3 - Check MK (if you use it)
+
+# Conclusion