Second Life UPS Mark II | Hacker News Reader

Second Life UPS Mark II

pop.fsck.pl

78 pointsproxysna82 comments

This project is a good reminder I really should finish that LiIon article that I've had on the back burner ever since I built the long range e-bike.

Some observations just from the pictures:

- corners of the nickel strips are not rounded, chances of cutting through the insulation of the cells, and in the high res pictures you can see some places where that might already be happening.

- welds are all over the place, from nice penetration to very shallow, possibly welds with high resistance, possibly burn through of the interconnects.

- physical construction of the pack is dangerous, no protection against impact if the whole construct is dropped. If it ever is dropped I would recommend immediately placing it in a fireproof container that is not indoors. And don't transport it in elevators or vehicles.

- no insulation rings between the tops of the batteries and the metal strips! That's really asking for trouble. You can see more than one spot on the pictures where the metal strip is directly in contact with the plastic and the welding process has likely further reduced the safety margin. The plastic sheathing of the cells is not enough protection for this kind of construction. Any insulation failure there means that you now have shorted an unfused pod and that pretty much guarantees a fire.

- balancing wires cross and pass over parts of the pack at a different voltage, a short of a balance wire can set off a thermal runaway.

- in general, wire management needs to be substantially improved if this is ever to become safe. Way too many wires that carry wildly different voltages cross and recross, especially close to soldering joints that might cause issues because the wires tend to shift inside the insulation if they're under tension while soldering.

I'm both impressed and horrified, this is the most clear illustration ever that I've seen for 'knows enough to be dangerous'. Laying out the sense/balance wires on a pack is a lot of work if you want to do it safely and even the pros (looking at you, Bosch) get this wrong. The result is that if ever one of the sense wires shorts out that it will essentially become a fuse and the energy imparted to the cells may just be enough to set off a thermal runaway. Wire management on the project here is terrible.

Nice comment. I was scrolling and going "ooh looks like a bomb", "oooooh", "oh yes thats basically a bomb".

If you like this kind of bomb then you should check out the various diy powerwall channels and blogs. Some of those I can't watch without yelling at the screen.

They're amazingly funny. Just the overall sense of "oh it's just a matter of time before this person loses their house in a fire." Yes mr. power, that small sandbag will stop the inevitable fire.

We put in solar this year, and my neighbor, who I used to think was reasonable, walked me through the MASSIVELY unsafe powerwall he installed after learning on YouTube how to do that.

Time to move...

I have a freestanding house for a reason. I've seen more than one place go up in flames because someone figured their attic gardening project would compensate for the mortgage.

Luckily we're country neighbors, so he's ~3/4 of a mile away.

I'll be able to see the flames, but other than that, I'm good. If he was connected to my house, or even close like in a suburb, I would absolutely report him to someone.

But because he's not, I just chuckle to myself and appreciate that he has no spouse or kids to die in the inevitable fire.

I think part of the problem is that on the outside Lithium Ion cells look no different than any other battery to a lay person. They probably assign roughly the same danger level to them that they would to a C-cell, rather than the conveniently pocket sized handgrenade that it really is.

A regular C-cell is already a small hand grenade in case of a fire. While not as dangerous as li-ion, they might a nice pop if on fire. (don't ask what I did in my stupid youth phase)

True, but they won't be the source of that fire.

Is this kind of thing covered by building / electrical codes?

It sounds like a domain where that would be sensible, given the potential risk to building occupants.

EDIT: For clarity, I'm referring to a power wall, not the Second Life UPS.

It's going to widely vary by local jurisdiction. In the US though, generally building codes would end at the recepticle. If this was hard-wired it would probably be an issue for the inspector, but otherwise it would be out of scope.

I expect UL would have something addressing this.

Homebrew stuff is not required to be certified by UL, in fact, surprise nothing is required to be certified by UL. But UL certification will have the effect of retailers marketing your products and reselling them for you.

But if you have fire insurance on your house, the odds are very good that your coverage will be void if you have non-certified things installed or plugged in.

Many homeowners have discovered this the hard way, and their coverage was denied even if the fire wasn't caused by that equipment. Just having it installed is enough.

Interesting! But fortunately it doesn't apply to me, and it would seem to me that if the fire wasn't caused by that equipment that would be a fairly easy thing to challenge. Because to deny coverage after the fact on a 'gotcha' like that would seem to be illegal.

It's legal because the restriction is explicitly part of the contract -- the insured party is promising that it won't use such equipment. Nobody is trying to deceive anyone here, the insurance company is saying it won't cover properties that are taking this easily avoidable risk.

My apartment building in NYC just banned all "lithium powered vehicles" (bicycles, scooters, etc). Building insurance is getting more expensive, and some insurers are pulling out of the NYC market altogether, because too many buildings are catching on fire due to Li-Ion battery packs ... it's always a bunch of cheap beat-up delivery bike batteries charging in a pile in some apartment, "unauthorized repair shop" or similar.

I'm kinda annoyed that this has been simplified to "no electric bikes/scooters/skateboards/anything anywhere on the property". Not all battery packs are the same ...

It's a coarse filter but they don't really have much choice, otherwise it would become such a nightmare of rules and subrules besides the inability to inspect quickly for violations. I keep my Lithium Ion based packs in a detached garage just to be on the safe side and I'd never bring them into the house, not even for storage. Besides the laptops and the phones :(

this being the Internet age, someone's gotta have built a lithium ion bomb intentionally and detonated it somewhere safe

My coworkers and I machined a stainless crucible and lit off a thermite charge on a pier in SF... didnt post online... so maybe Im making it up... XD

oh LOL never thought of that, its like they are a virtue signaling crack house!

Well... yes. There are amateur projects and then there are amateur projects with a little bit of "spice" on it where you have to grow up and mature with your process just a tiny bit.

For example anything having to do with AC or rechargeable cells goes in this category in my book. I also throw in sensitive electronics that needs a little bit of ESD safety.

Electronics that aren't ESD safe will not burn down your house though. This just might.

Yeah, ESD safety is just a yield management strategy. "It sure is expensive when every 100th board comes into QA dead." Being careless with lithium ion batteries is "your house burns down, and your insurance company won't pay for it" levels of dangerous; possibly the largest financial risk you can take on as an individual! It's also bad if your family dies in a fire.

Personally I just stick with commercially-made solutions from reputable brands here. You want someone your insurance company can sue when it blows up!

I have never had any bad luck with lithium ion / lithium polymer batteries, but Reddit randomly shows me r/spicypillows from time to time, and ... it's pretty scary. What would you do if your laptop starts blowing up like a balloon? Have a plan!

Carrying around a bag of sand to throw your laptop into is wildly impractical though. What's your plan?

Let it do its thing outside.

If I were going to do a DIY UPS, I would try to use lead acid batteries designed for this exact purpose, and install them in a room per the manufacturer specifications for ventilation and relevant charging interlocks. Not sure why we would want to chase miniaturization as a design criteria for something like this application (I.e. "rack mount").

Lithium ion is accessible and dense, but hilariously unsafe as you scale up in a DIY residential setting. I won't even get official, first party lithium ion backup power installed in my home out of fears it could cause a catastrophic fire. I keep a 2kwh lipo pack on a concrete slab behind my house for emergencies. If you are storing these things in your home, you need to do some napkin math regarding what 2000 watts for an hour could do to your home if released over a much shorter timeframe.

I have been in a few "DIY Powerwall" groups which usually source the cells from old laptop batteries and this community is very aware of the fire danger so they come up with hilarous solutions for their home racks like having plastic bags full of sand above the batteries so when they catch fire the plastic melts and the Lithium cells get covered in sand.

Most people do try to have the cells on the side of - or inside outdoor metal sheds to minimize the fire damage

I think the entire premise is cursed. That is to say, if you are finding yourself with a need to store more than maybe one KWh of energy in a residential setting, you might need to reevaluate your overall architecture.

A generator, while not ideal ecologically, is a perfect abstraction in engineering terms. Especially generators connected to pipelines. Those are the real win in safety - the energy is only received on site the exact moment it is required. Some trade offs with resilience but you don't have to go that far. On site diesel and LPG have proven safe and reliable as well.

For me the batteries are just about transferring the load. If you need run time, you should find yourself a generator. For non emergency, non-UPS scenarios, sure you could maybe plan an off grid setup. But I think we'd be mixing use cases a little bit with this angle.

Beyond a certain point, I don't think there exists a safe way to store certain embodiments of energy on site. Sandbags or no.

I can see the other side of it too: some people would like to be carbon neutral one way or another, and with net metering laws under attack everywhere (because the government would like to tack on tax on your own generated power) DIY powerwalls are an efficient alternative if you don't look at the risks involved.

Once you add risk into the equation it gets a lot more complex in a hurry. But that's the main driver behind all of these 18650 recycling projects. I built a fairly large pack for an e-bike (170 cells), the batteries cost some money but the peace of mind that gave was well worth it to me. But I actually can afford it. If you can't it isn't long before you realize that electronics are cheap but cells are expensive so if you can get a cheap source for cells suddenly you can build that powerwall. But to do so safely will more than offset the money saved on refurbished cells. And the workmanship requirements are also not exactly easy to come by and all it takes is a little mistake to have a fairly spectacular result.

Very true and let's not forget the DIY Powerwall people are the real recyclers of old batteries. They are also the ones who ae in front of all auction houses that sell batteries of totaled EVs and give them a second life.

I think neighborhood batteries make a lot more sense - spread out the cost, if every neighborhood had a reasonable sized battery installation, the grid would be FAR more stable.

Utility scale batteries Someplace outside the city. It doesn't really gain much to put them in a neighborhood and costs a lot more. When you are running on battery you need oversized inverters as you need to cover the worst case of the whole neighborhood. The more area your battery covers the less overbuilding you need because the closer you are to average. House scale battery works better (but has obvious safety issues) because if there is a power outage most people can shed a lot of load without discomfort and thus get by with must smaller batteries.

My home lpg tank (800 gallons) is far enough from my house that even if the worst case where the whole tanks opens up and starts on fire (I don't see how this is possible, but that is the only way I can think of to get enough O2 in the tank for a fire), my house is still safe.

A home powerwall that was similarly far form other buildings should be safe enough. Look at utility scale battery installations and it is clear they have a lot of separate units, so fire in one won't spread to the next (or at least the fire department will be able to put it out before it spreads to more than a couple). However most people don't have that much space in their yard.

> you need to do some napkin math regarding what 2000 watts for an hour could do to your home if released over a much shorter timeframe

The energy released in a lithium fire far exceeds the electrical capacity of the battery.

A few people here do mention LiFePO4 batteries as a safer alternative.

As a total battery-noob, honest question: assuming the author had indeed used LiFePO4 for the build, with standard (-ish?) over- and under-charge protection. How much safer would this be? It's still a relatively big battery pack (read: chemical energy) sitting in someone's broom cupboard, isn't it?

A lot because they suffer less from thermal runaway and are more resistant to physical damage. That's why the better tool brands use that chemistry too and why you see car manufacturers switching to them.

I think LiIon was a huge mistake and that as various pieces of gear that have them built in ages that we'll see more and more issues with them. They're everywhere: cameras, laptops etc, stuff that has a service life of 10 years, sometimes more and as they age they become more dangerous, especially when people re-purpose them.

I've rebuilt a couple of packs for brand name cameras and e-bikes and it is very interesting to take apart the old packs to see what you find inside. More than one had the start of a fire in them, evaporated wiring, scorch marks and so on. I'm pretty sure those are the good ones ;)

Dense energy storage is almost inherently dangerous. LiIon are great for applications where you actually need that density. But the scale of those use cases has made it ubiquitous and cheap, leading to LiIon to be used in many cases where the density isn't needed at all and is just a pointless risk.

The concept to research here is thermal runaway. There is a lot of chemical energy in wood, too, right?

Yes but generally if I’m rough with a piece of wood it doesn’t start a fire.

Depends on how rough ;)

Please don't fuck around with 18650s if you don't know what you are doing. And unfortunately the author doesn't. The main fuse is a necessity and the thermal fuses are a neat idea but ESPECIALLY when using repurposed cells, cell level fuses are a must and they must be calibrated to account for the higher internal resistance of reused cells. The author also only briefly mentions the necessity of matching cells and measuring internal resistance. This is not something to casually brush over. Also, for UPS usage, most 18650s are not the right kind of chemistry. They don't like to sit at high charge state for extended periods of time. Use LifePo cells for that. They are much more forgiving and significantly less likely to murder you in your sleep.

One 18650 going into thermal runaway is bad (ask me how I know...). A pack of 18650s you causally fuck with can easily burn down your home.

Wanna see some results of that? https://endless-sphere.com/sphere/threads/dogman-dans-e-bike...

TL;DR: high energy density batteries can cause high energy rapid unplanned discharges. Please don't fuck around with batteries unless you know what you are doing (you don't)

I wish we could have UPS systems that could take large bring-your-own 12v size LiFeP04 batteries. The ones with terminals that look like car batteries.

Fact: All Battery powered devices should have easily user replaceable batteries

Ahhh but that sweet incompatibility driven mark-up...

Don't worry, the market will self-regulate. Any day now! :)

It's happening:

https://www.cordless-alliance-system.com/

and

https://www.powerforall-alliance.com/en/

Of course there had to be multiple efforts...

ayayay, that reminds me to check the battery of my ~2008 Bosch :D

Yes, you should. But, even if the construction of the Bosch packs is not up to their usual standards (especially the pannier mounted packs are terrible from a water ingestion POV) the BMS is top notch and it tends to shut down the battery if it detects any serious problems, far better than most el-cheapo BMS.

Look for any traces of moisture on the boards and the cells and if you spot any it is probably safer to discard the pack or have it rebuilt. The seals need very careful attention, less so in the frame (vertically mounted) packs. The powertubes are much better from an engineering point of view.

Most any UPS that has a user replaceable battery can be swapped to 12v lifepo4 batteries. they won't likely fit, but I took the two 12v lead acid batteries out of my UPS and extended the wires out the back and to a new shelf that holds two 12v lfp4 batteries. You should get 12v lifepo4s that have built in BMS to protect them from over/under charge. Mine works great and I added a lot more watt hours to it. The only thing that won't work well is the battery level percentage indicator since lfp4 don't read the same as lead acid.

I have an older (cheapo) UPS that I just replaced the battery on, which is probably over 15 years old now. It was much easier than I thought.

Coincidentally, right after I replaced the battery and left it to charge overnight, it saved my workstation from a power outage [events which aren't too common in my location] the very next day. It was quite satisfying to be able to save my work and shut down properly.

The old battery had stopped working long before replacement. So I guess the UPS was no better than a surge protector at that point.

These exist, just look for inverter/chargers. Tripp-Lite APS750 as an example though may not be LiFePO4 compatible. I use one as a sump pump power backup.

Various Victron products as another example that are actually LiFePO4 compatible: MultiPlus, Quattro, or others depending on what capabilities you need.

They have them, they are called inverter-chargers. I recommend Victron or Magnum for quality with UPS grade switchover.

I keep wanting to do a Victron 3000 12v for a critical loads in my house (fridge, lights, outlets) to use with some spare 6V lead acids I have I took out of my RV when I switched it to LFP. In the future I could switch the batteries out for LFP if I wanted.

If building from scratch I would probably do a 48v rack style LFP with 48v inverter instead again probably Victron, wish they would make a rack mount version.

Victron is good stuff, seconded. Used extensively in marine applications, I've yet to see a unit fail that wasn't purposefully abused.

> they must be calibrated to account for the higher internal resistance of reused cells

How can the cell tell whether it is "reused" or not?

If it is true that the cell-level fuses must be calibrated to account for the fact that the cells are reused, would it not also be true that fuses must be recalibrated after some time even if the cells remain in their original application?

> Please don't fuck around with batteries unless you know what you are doing (you don't)

The "you don't" part isn't helpful, because it encourages a worldview in which technology is magic that can not be understood or harnessed.

They didn't say you couldn't learn, they said don't fuck around. That seems like pretty reasonable advice to me.

I wasn't objecting to "don't fuck around with batteries", I was objecting to the unequivocal "you don't [know what you're doing]".

I wrote it that way because the hacker ethos here makes a lot of people assume they know more than they actually do about all things tech. It's easy to learn how batteries work and how to make big packs of cells but the practical and institutional knowledge required to do it safely is not something one should just "hack" into. That's why.

I totally second that wording and I do know what I'm doing.

This setup is only safe in a structure that doesn't have people or other stuff in it and that you are not going to be upset about if it burns down.

> If it is true that the cell-level fuses must be calibrated to account for the fact that the cells are reused, would it not also be true that fuses must be recalibrated after some time even if the cells remain in their original application?

If you are being conservative, the fuse is already well below any peak ouput a cell might give you so aging cells are considered. Also in large packs, cells should (in theory) age roughly equally. But If you match a LG HG2 and a Sony Conyon and both fuse them for 50A and then wait a few years of usage your gonna have a bad day

> The "you don't" part isn't helpful, because it encourages a worldview in which technology is magic that can not be understood or harnessed.

An essential part of that is actually understanding it and knowing what your limits are.

When you do extensive research, understand all the risks involved, and seek help to review your work, it is definitely possible to do projects like these - even as a hobbyist. The problem is that this looks to be closer to "hold my beer while I grab whatever cheap module I come across on AliExpress".

I design electronics for a living, and I personally wouldn't want to touch a project like this with a ten-foot pole. The risks are just not worth it. There are plenty of other fun things to do which aren't going to burn down my house.

It's extremely helpful! It helps keep the author of TFA safe.

The "you don't" is leaning on the idea that if you actually know what you're doing, an internet comment saying you don't isn't going to dissuade you. If you aren't confident enough to ignore an unequivicol "you don't understand this", then you definitely don't know enough to mess around with this. (The converse, of course, isn't true. There's always going to be someone who is still overconfident enough to ignore it even without having enough knowledge).

This is spot on and if you don't understand why then you are out of your depth.

If the author reads this: decommission your project, please and safely destroy the cells. If you really want to do this project drop me an email and I'll help you as much as possible to do this safely (and with a different chemistry).

Matching cells is hard and cells that have been through a different life cycle, especially cells that have been turned off for safety reasons won't stay matched for long. That is because their internal make-up will be different and so they will respond differently to charge/discharge cycles with one or more of the cells taking the other ones with them. And please use a FLIR on the pack to determine if there are cells that are already 'hot' compared to the rest. That's a sure sign that something bad is brewing.

All it takes is one degraded cell and this project is a fire waiting to happen.

Bonus Points for an UPS that will keep them charged at 100% permanently.

Yes, that's why LiIon simply is the wrong choice for this project. But I can see why they did that: they are readily available, much more so than LiFePo4.

From a technical perspective it's an impressive project, but geez that's a lot of cells being packed together!

Especially with them being no-name refurbished bargain-bin cells and the very minimal literally-dollar-store-quality protection added, I'd be scared shitless about this catching fire sooner or later. If any one of those 96 cells goes bad you've suddenly got a basically unstoppable inferno in your home.

I mean, you do you, but I sure wouldn't be happy if my neighbour was doing projects like these in their garage!

Mount it in a waterproof box outdoors, and just expect that sooner or later it will catch fire, but when it does nothing will be damaged.

That's what I did with my DIY solar batteries - and so far they haven't caught fire.

You still need thermal management if the temps can go near or below zero.

Depends if you care about longevity...

True, but charging while too cold can also damage the cells in ways that are not immediately visible and that in turn might trigger thermal runaway rather than just cell capacity degradation. That's because the cells on the outside of the pack will likely be much later to the party than the ones on the inside during charging. But any good BMS will simply refuse to charge as long as the pack is too cold.

There's entire communities of DIY energy storage hackers (like r/DIYPowerWall). You'd usually house these contraptions in a freestanding shed in your backyard, far enough from anything flammable, and monitor the thing very closely (voltage, temperature, ambient sensors, etc). Some of the pictures look like home-made starship interiors.

(Edit for clarification: I do not necessarily endorse this kind of activity.)

Unfortunately they don't spend nearly enough time explaining all of their decisions and lots of the related youtube channels have a following that is not in line with the safety of the advice provided.

Freestanding structures are a great way to ensure that if - or should I say when - it goes that at least it will only be a monetary loss.

There are some youtubers that are basically setting people up for an accident without any indication of the danger levels involved. To be fair: the same goes for some professional gear that you can just go out and buy and which people routinely charge in their hallways and attached garages. E-bikes, gardening gear etc.

I guess it depends how far the garage is away from my home.

If anyone is considering attempting this, please ensure you first educate yourself about thermal runaway.

Using a collection of old lithium batteries like this can be extremely dangerous, and without exaggeration, it can kill you.

When (yes when, it will happen) one battery overheats and catches fire, it will cause the others to do the same, leading to a significant fire. During combustion, these batteries also emit gases, so put this into a box and you risk an explosion on top of the fire.

> please ensure you first educate yourself about thermal runaway

YES!.

That's why you should monitor a setup like this with a FLIR from all sides during the first month of operation or so, if no batteries ever run hot during charge/discharge cycle you may be good to go. But better keep an eye on that balance, especially in packs that are sitting at high voltage for a long time (like this one does) because you won't know how the pack behaves during discharge and recharge and that is likely when any problems will be most apparent.

Thermal runaway can happen very quickly once a certain threshold has been passed.

If you really want to do a project like this with Lithium Ion and you are not space constrained you would do better to have a bit less capacity and charge the batteries to 3.6 ... 3.8V or so. That would work much better in the long run.

Reply on news.ycombinator.com