Hacker News Reader

Comment by lloydarmbrust | Hacker News Reader

I've spent about 1,000 hours on this with my team using a very expensive (~$250k) lab and I'd love for this to be a great resource for people when choosing a mask.

The heavy lift we are doing here is around our particulate filtration efficiency test in which we do either the NIOSH N95 NaCL (0.3 micron) test or the ASTM 0.1 micron latex particle test.

I'd love HN feedback on how we improve the interface and make this a true resource for people.

Disclaimer: Yes, I'm a mask manufacturer and that's how I justify keeping all of this equipment up to spec, but I'm not big enough to make all of the masks for the world so I'd rather people find a good mask than just use my mask.

Any feedback is appreciated.

View on HN

Your filtration test appears to be only testing the mask fabric, but not the fit.

I see a number of surgical masks over 95%. I don't think that would be possible in practice, because of leaks at the sides and top. I'd expect more like 50%. But, you don't appear to have the equipment to do a fit test.

Refer to the 3M video on how to fit test an N95.

I realize that N95 and surgical masks have different standards, but you are making them look comparable, when they are not.

A good mask needs both fit and filtration... most people can put a mask on, check for air gaps and see how well it fits, but they don't have a $250k mask lab to make sure the fabric is good.

We are just solving for the filtration and letting people figure out the fit.

But, you make a good point, in the videos I try to point this out but we could do better. Where could I add this to make this more clear?

You could start by adjusting your given filtration ratios by predictable leakage based on the type of mask. It's wholly misleading to rate the filtration potential of an earloop mask as equivalent to that of a 3M Aura.

Not exactly.. it really depends on the situation.

For example an ASTM 2 or 3 surgical mask fabric has a filtration of 98%+

On my face I have a leakage of about 21% or PFE of 79% when worn normally.

With an ear saver, it jumps to 91%.

With a mask brace, it jumps to 98%.

So it really is about the fit.

If I put leakage % it would vary from person to person and many people would be mislead by this.

> On my face I have a leakage of about 21% or PFE of 79% when worn normally.

> With an ear saver, it jumps to 91%.

> With a mask brace, it jumps to 98%.

I had no idea it could vary so much for surgical masks. I’d be curious what range of leakage people could typically expect for other designs such boat-shaped KF94 or flat-fold KN95 masks.

Fit seems important enough to deserve a mention in the intro paragraph at the top of the home page. A more detailed explanation, including the example you gave of how much it can vary for even one person, could go in the “Click here” link to the full testing methodology.

Thank you for doing this, by the way, rare to see a manufacturer test so far beyond their own masks. It clearly takes resources to do this and is a great service to all of us who have been relying on guesswork.

Okay I put a disclaimer right on the front and we are typing up an explanation right now. Thank you for this feedback!

https://www.armbrustusa.com/pages/mask-testing

thanks! all the notes about fit will be very helpful, including how to use the extra accessories you mentioned above.

For example I found this study on beard length which made me reconfigure my facial hair: https://www.nature.com/articles/s41370-021-00337-1

The charts in that paper also make it seem that n95 *typically* has a much better seal on the face which is worth noting.

I also wonder if the breathability you're measuring affects filtration, given imperfect fit. I would imagine that worse breathability would cause more air to flow around the mask rather than through it.

How did you determine those percentages?

Using the industry standard TSI PORTACOUNT® RESPIRATOR FIT TESTER 8048

It measures the air outside the mask verses inside the mask.

https://tsi.com/products/respirator-fit-testers/portacount-r...

Do you have the same measurement for you for an N95 with the typical elastic straps that you can share?

Yes but it differs from model to model.

I get as low as 80% and as high as 99% on my face.

Fit is a big deal because it's personal and hard to communicate. I think it was Amazon that found when scanning their workforce that they discovered 29 different face shapes!

> but they don't have a $250k mask lab to make sure the fabric is good

Hard to argue against that, but on HN I'm sure someone's gonna try :-)

Someone on HN recently did:

https://news.ycombinator.com/item?id=29878971

hahaha got me on that one.

You could add a clear statement to the testing procedure popup that you are measuring the fabric only, and that (except for N95) the filtration of a mask is strongly dependent on fit and leakage.

Okay working on this now.. thanks for the suggestion. I should note that even N95 is extremely effected by fit. I've gotten as low as 80% filtration on a flat-fold N95.

Unhhh, fit and leakage matter a lot for N95 too.

Done

I think it should be made way more clear - and here I take the opposite stance of the person you're replying to.

Change title to "Comprehensive Mask Material Testing" or "Comprehensive Mask Fabric Testing" or similar.

Similarly; "...that's why we're testing the filtration capabilities of every mask in our laboratory"

At some point (on the front page, above the fold) there could be a small notice box explaining the difference, or ?-link or tooltip to read more.

I think a lot of the suspicion you're facing in this thread wouldn't have been there if it weren't for that ambiguity, and the need to go into deep content to resolve it.

For someone who just browses to your site and spends 5 min finding "the best mask", they would never realize as it is today.

Okay I put a disclaimer and typing up a post to link to LMK what you think: https://www.armbrustusa.com/pages/mask-testing

Thanks!

https://www.armbrustusa.com/blogs/news/mask-fit-vs-filtratio...

Oh, excellent update!

(It might be a good idea to break your database into two sections: one for N/KN/KF respirators and one for other masks.)

I also notice that you have a "collections" page

https://www.armbrustusa.com/collections/

That page has a very helpful section, "USA made N95 masks and KN95 respirators".

https://www.armbrustusa.com/collections/usa-made-n95-masks-r...

Exactly what we need. Two questions:

1. Aren't all N95 "masks" actually respirators? 2. Why do you offer "made in USA" KN95 (Chinese standard) instead of an Armbrust branded N95?

Thank you for doing this!

Hey, I think it's a lot more clear now :)

woot

These are good points. Working on rewording this now.

Perhaps you could add a “fit ease” row to your scoring? Masks with head straps could have a check and others could have a ? or something.

yes i could make a mask out of plastic film and it would have 100% efficiency

this probably explains why in these results, most adult masks have a pass rating, but most kid masks have a fail rating

Great work, thank you for sharing it with the world.

I am very curious about country stats and have 2 questions.

First, I found this one randomly: https://www.armbrustusa.com/blogs/mask-review/kf94-try-mask-...

The country of origin on your page says China, but the photo's box says Korea. Is this a mask repackaged and exported from Korea that is actually from China?

Second, does "China" also include Taiwan? If so, I would be interested in separating them for statistics.

for those interested, here's a first level breakdown of pass/fail by country

  | country   | # pass | # fail | pass rate |
  |-----------+--------+--------+-----------|
  | USA       |     39 |     27 |        59 |
  | China     |     48 |    186 |        21 |
  | Korea     |     31 |     13 |        70 |
  | Thailand  |      0 |      2 |         0 |
  | India     |      2 |     12 |        14 |
  | Mexico    |      1 |      0 |       100 |
  | Germany   |      0 |      1 |         0 |
  | Singapore |      1 |      0 |       100 |
  | Vietnam   |      1 |      0 |       100 |

also, in this dataset it looks like adult masks have a 66% pass rate as a whole, while kids masks have a 73% FAILURE rate!

So a third question is, are you using adult mask testing methods for testing kid masks as well (which would explain the failure; on the other hand, the kid masks that passed must be quite some mask!), or do kid masks just suck in general?

In the US the standards for adults are the same as kids... in China they just came out with a new standard but for reasons I wont go into we don't cover it (very China specific).

That said, I have also noticed this... if a manufacturer in China makes a great adult masks, their kids mask is likely to fail standards for filtration AND be harder to breath.

Having made both adult and kids masks I can tell you that there is way less of a demand for kids masks and because of that less attention gets paid to them on the manufacturing floor. Theory: People are just lazy and make worse masks for the kids I think because kids are not paying attention.

Furthermore, kids don't care that it passes standards, they care if there's a dinosaur on the mask or not. Even my own kids tell me my kids masks are boring! So manufacturers put their time on making it fun and not on protection.

We test both kids and adults to the same standards, use the same fabric, etc. Boring but effective :)

Ah! You caught a mistake. Thank you it's fixed.

China and Taiwan are separate (they make very different masks) but I haven't been able to find any from Taiwan to test.

thanks for the clarification!

(apologies, I edited my original message with a third question, which I would love to get an answer to, but now I don't want to mess with this thread anymore)

I read through your "methodology", and it seems to be too simple to explain your pass/fail results. For example, you fail this mask: https://www.armbrustusa.com/blogs/mask-review/kn95-arun-flat... even though it seems to meet the filtration and breathability standards, but is still listed as having failed.

I guess maybe you explain the failure in the video, and maybe there's a good reason that it failed, but you're asking me to waste a bunch of time watching videos to understand information that could have been provided in print. It doesn't make me trust you.

Read their methodology,

> A good example of this is in the text written on the front of N95 and KN95 masks. The standards for each of these masks demand that very specific information be printed on the device. If a KN95 mask does not have the manufacturer name and the standard number they are meeting (GB 2828-2019 or GB2828-2006), then they cannot be considered KN95 masks

Such masks are marked failed. My guess is thats what happened here.

That doesn't seem to be the case here, the product images clearly show the required labeling and it's even pointed out specifically in the video review [0] at 3m30s.

Would be good to see what the update notes say for this one. It might be because the product listing still claims it's approved from emergency use; the FDA rescinded the EUA for all non-N95 masks back in July [1].

0: https://www.youtube.com/watch?v=zeYlKgOI8-w&t=210s

1: https://www.fda.gov/medical-devices/emergency-use-authorizat...

I disagree with this methodology. At the very least it should be called out as the reason for the 'fail'. Yes, it's less than ideal to not follow the standard. But I would far rather an N95/KN95 mask that didn't have it correctly printed but otherwise passes the criteria than none at all.

If it's a filtration test and it filters correctly give it a pass with an asterisk.

Yes the reasons for failing are in our testing notes being transcribed now. Over 250 hours of actual testing just to get to this part and with Omicron we wanted to get the raw data up ASAP.

Notes should be live in about 275 of them hopefully today.

It's a nitpick. I'm glad someone is doing this.

And it gets worse when you look at the page on mobile, miss the entire filter percentage column and that the list is ordered by that: then all random samples reached without excessive scrolling are mystery failures despite good filter result. Really strong example of how a seemingly harmless presentation failure can taint the perception of content.

We have updated the mobile view please LMK what you think.

Much better and nice to see that the change didn't compromise the desktop version.

Still kind of unimpressed by the high weighting of labeling formalities for pass/fail (I live in the realm of EN149 anyways) and I'd certainly prefer a few notes over video. Feature request for the next pandemic: I'm sure people wearing glasses would love being able to filter for masks with that spongy cell foam flow blocker strip on the underside of the nose stripe?

But I really love the inclusion of breathability: I'm convinced that in the day to day realities of living through a pandemic, a mask with better breathability won't just be more comfortable than one less breathable, but also provide more protection, because it's worn better and more. And a list like that would have been very helpful earlier in the pandemic not only for buying guidance, but also for instructing people that if they really can't stand wearing a mask, maybe that's in large part because they simply haven't encountered a good one yet. Seeing a range from <50 to >200 and how little it correlates with filtering (it does, but much less than one would expect) is worth more than words. I didn't realize the breathability factor myself until I found myself in a hospital abroad (injury, not infection) where they supplied me with proper FFP2 for going to x-ray and the like. So much better than the emergency use 3-ply crap I used before and almost decadent indulgence compared to the emergency textile contraptions that preceded them.

Greetings from Germany. Of course my local interest lead me to the one and only product tested that originated here.

https://www.armbrustusa.com/blogs/mask-review/surgical-medte...

The accompanying video is informative and I can add that the way the earloops are attached is wide-spread in Germany. Also confirmation from my side that the strings rip right off with this type of attachment regardless of the actual manufacturer.

On an iPhone, the value for color in the details is slightly too far to the left and in this case wrong because it reads Black. I’d like to be able to tap on the color and see a list of matching masks and also have a top level filter for color.

As a parent I’d love to have a top level filter to see masks for kids.

Finally, German quality engineering exists but it’s really not found that often in the wild as the myth would lead someone to believe. So thanks for helping to debunk that in the video.

>Finally, German quality engineering exists but it’s really not found that often in the wild as the myth would lead someone to believe. So thanks for helping to debunk that in the video.

I don't think you should base anything on N=1, either for or against.

If it helps 100% of my grandparents are from Germany :) might be obvious by my last name.

Crossbow. Yeah, you mentioned in the video. :)

Table only shows mask names on mobile. Don’t be surprised if your announcement doesn’t have the impact you wanted it to. You need a website that does its job on mobile.

Okay we updated the site to work on mobile. Thanks for this... seems obvious now but total miss on our part.

Thank you!!

Ahhh... good point. The search will work but you are correct that the full version is better on desktop.

The GP has a good point - I was about to write a long HN reply about being able to sort or show the “best” masks. Saw the “list doesn’t show scores” on mobile led me to this comment.

As a consumer, when I land on that site, “what’s the best mask I should buy” (protection level, budget, comfort, local/globally made). If it’s not clear on mobile, it’ll be pretty hard to get non-tech people to stay on page for more then 10s or funnel them to the page you want.

Even with "desktop mode" enabled on mobile (android chrome) I still see only one column. You can click in for more information but it is pretty useless as an interface.

I think this is great and thanks for doing this! I have three small suggestion:

1. For masks that fail on marking issues but not filtering efficiency, could they get a third category that’s not pass or fail (something like ‘non-compliant’)?

2. I wish you could filter on pass/fail

3. I wish you included/could filter on price per box and price per mask.

Also big thanks for doing this from me and a third suggestion:

I wish I could filter for "valve" because valved masks are not allowed where I live.

My question is after weeks with the same mask… how does it perform… also bonus what kind of additional germs to I get to experience close to my face

You are not supposed to keep the same mask for weeks... They are meant to be replaced after hours.

Sorting by breathability is broken.

It'd be nice if you had pop-up dialogs upon hover over each of the fields. For example, if I hover over breathability, explain what it is and how you measure it.

Some color codes/gradient would be nice to help with the visual effect (Green for Pass, Red for Fail etc.) But please do include a toggle for color-blind mode if you do add color.

It's working for me, can you describe what's happening?

Good call on the color coding... I'm color blind so didn't think of this :)

Interesting, it seems somewhat fixed now.

Initially, I was seeing the sort return 1, 10, 100, 2, 20, 200.. so sorting by the first numeral instead of the number. But now it seems to be sorting ok-ish except if you jump to the bottom and the row for "Staples / Beixikang Face Mask" shows a % symbol for some reason. So possibly some data entry issues.

Below the Beixikang, there are then about 12 entries in 80-168 range so the entry for the Beixikang may be throwing the sort off.

When I sort by breathability (Firefox desktop) it goes 9 up to 572 for most of the list then '79.46%' then 80.2-183. The other way it goes 162 down to 9 then '79.46%' then 572 down to 80.2.

> Sorting by breathability is broken.

I'm seeing this too; I think the single entry [0] that includes a percent sign in its breathability score is screwing up the sort.

0: https://www.armbrustusa.com/blogs/mask-review/surgical-beixi...

> Some color codes/gradient would be nice to help with the visual effect (Green for Pass, Red for Fail etc.) But please do include a toggle for color-blind mode if you do add color.

If the green and red shades are chosen to have enough contrast to look different when convert into black and white (e.g. when printed), it should be usable even for most colorblind people, while also being easier to read for everyone.

Nice! Fixed

I looked at the linked page and it had a big list of masks, each of which linked to a page about that mask with a picture but no information such as test results. Was that intended? Could you just ditch all the fancy web design and have a big table with test results for each mask, with an ordering link for the models that you sell? That would be awesome.

It would be great if you also sold the stuff you use for testing masks. I spent a while looking into testing masks myself, but then the shortage eased up so it seemed less important.

Testing masks is very specialized and all in I've spent about $250k on the mask lab. I just bought a new machine that costs about $90k. Here's an overview of the machine: https://www.youtube.com/watch?v=vpjajJrDWBk

Thanks, I'll watch the video when I get a chance. My idea was to basically reproduce the setup in Joel Amilcar's PhD thesis (https://ir.uiowa.edu/etd/2002/) using an off the shelf air quality sensor (PM2.5 with the hope of being able to extrapolate below that size), using a salt spray nebulized with vape gear. I didn't have any good ideas about how to control the particle size (this isn't my area at all) so had hoped to get some advice at some point (let me know if you have any thoughts). I had remembered reading about a guy on Youtube who did some tests like that but never managed to find the channel. Maybe it was yours! Someone else here mentioned Aaron Collins so I will have to check his videos too.

I think it would be great to have easily reproduceable, low cost test equipment available at shops and clinics because the quality of masks (even of a specific make and model) is likely to vary between units. You might find a good mask with your tests, and then find that your next order of the exact same mask is crap. So you have to keep testing. Maybe it is less bad now. I was interested in this more when masks were in very short supply, but then I managed to get a few boxes of them, and set the testing idea aside.

> I looked at the linked page and it had a big list of masks

All test results disappear when the browser window isn't wide enough (eg. on mobile, or narrow PC windows).

Being able to filter the list by pass/fail or minimum PFE and/or maximum breathing resistance would be nice.

Adding this to the feature list

Perhaps interesting for you: The University of Delft has done research on mask fit. A report in Dutch can be found here[1]. Perhaps auto-captions in English work for you?

[1] https://www.youtube.com/watch?v=zJZBNwa6QMk

Is this a mask testing company or a mask selling company?

How could I possibly trust one company to do both?

Hi, thanks for doing this. 1 piece of feedback is to have a couple lists of masks that you recommended so we don't have to comb through all the listings. I would put this page as the front page and then link/refer to the list to provide authority/weight to your recommendations.

Eg by type (surgical,duckbill,n95,kn95), by budget (budget, price/performance, premium), ships now on amazon, doesn't ship now but better in some way if you're willing to wait, etc.

Also I was looking at the 3m 9210 that you have 2 entries for in your table. The breathability is quite different between the two samples (98 vs 242). Is this expected? That seems a lot of variability for the same mask.

If you're open to taking "requests", I'd love to see this Australian one tested. https://amazon.com.au/gp/product/B093B8QKVP/

Good N95 style masks are difficult to get hold of in Australia. A few places are either charging absurd prices (like $10+ per disposable mask), but most pharmacies just don't stock them.

The one above has some fancy photos of certs in its listing, so it would be interesting to see if it's genuinely good. Happy to send some to you if it's helpful (you can dm me on twitter listed in my profile).

I see a ton of KF94 results, may be easier to find them due to proximity to Asia? They are supposed to be more comfortable than the beak-shaped kn95 anyway, and offer the same protection.

Might be, but it's hard to trust any of those listings - where they come from and if the KF94 really means anything at all (some reviews indicate as much too, but realistically most consumer have no way of knowing). The one I mentioned stood out as it's locally made and supposedly locally certified.

The biggest issue with surgical masks is most of the nose strips are pathetically thin so they bend out of place and stop you from getting a good seal, which means they're a nightmare for glasses wearers.

> which means they're a nightmare for glasses wearers

More importantly, it greatly reduces their effectiveness. Fogging up glasses is, by far, the lesser issue of the two.

Depends on whether you're the type of person for whom fogged glasses means a mistyped line of code or the type of person for whom fogged glasses means blowing up a lab or getting your arm ripped off in a machine.

Yes. Look for masks with a double nose wire. We make one and many find it does not fog their glasses... but there are plenty of others that make them too.

Hi, was interested to see this, but the video is just of you buying them? And the methodology link is to https://www.armbrustusa.com/pages/ which doesn't seem right, and 404s anyway.

From the comments here it seems others have got to the right content, so I'm not saying 'avoid this spam nothing here' or whatever, just not sure what's going on. Accessing from Europe (UK) if that's something to do with it.

I sorted the list by breathability, filtered out the failed ones, and only focused on the n95/kn95/94 masks (as breathability rating for surgical masks does not make sense, because it is not supposed to be tight), and the top 5 were:

https://www.armbrustusa.com/blogs/mask-review/n95-gerson-duc...

https://www.armbrustusa.com/blogs/mask-review/kf94-dr-puri-b...

https://www.armbrustusa.com/blogs/mask-review/n95-honeywell-...

https://www.armbrustusa.com/blogs/mask-review/n95-indiana-fa...

I was thinking the same. Do you have a video of you testing some of them as well? It would be interesting to see how it works!

Click on any individual mask, and it takes you to a separate page. On most of those pages, there will be a 5-7 minute video showing that particular mask being tested while Lloyd talks about the features of the mask.

I’m also not seeing U.S. Mask on that list. The thing I like about their KN95 style mask is that each one is individually packaged, unlike almost all the other masks I’ve seen.

https://www.armbrustusa.com/blogs/mask-review/surgical-beixi... has breathability expressed in percentage. Others not.

Buy this mask button should be green :)

The viewer should be able to select only those masks that pass. They should also be able to select a maximum breathability score. They should also be able to select minimum PFE rating.

I’m not seeing AirPop masks on this list, see https://www.airpophealth.com/us/home

Thanks for putting in the effort and publishing the data. Are there indications that point to good/bad filtration quality? What do the good/bad products have in common?

Um, Your site appears to be completely broken - once I scroll down once scrolling (up or down) no longer works at all.

My budget is limited, but I just purchased a $50 shipment of your masks because this is awesome.

Thank you.

Thanks! Honestly didn't do this to directly sell masks... according to shopify we have made a whopping $248 from this post

List doesn't seem to scroll in Safari desktop :/

on the 3M aura, "made in american". otherwise great site

It actually depends on the model.

9205 is US made, the 9210 is made in Singapore.

Did you fail all the Chinese masks?

Reply on news.ycombinator.com