Hacker News Reader

Comment by IshKebab | Hacker News Reader

This is really great. OpenTitan has some useful IP components that can definitely be reused, and it's really cool that this is open. Nice one Google. I have to minority nitpick though:

> both individual IP blocks and the top-level Earl Grey design have functional and code coverage above 90%—to the highest industry standards—with 40k+ tests running nightly

This is definitely not "to the highest industry standards". I've worked on projects where we got to 100% on both for most of the design. It's definitely a decent commercial standard though - way above most open source verification quality.

View on HN

You can see the latest nightly results here: https://opentitan.org/dashboard/index.html note there are some 100% figures.

Having spent several years working on OT I can tell you that most of the gaps are things that should be waived anyway. Getting waiver files reliably integrated into that flow has been problematic as those files are fragile, alter the RTL and they typically break as they refer to things by line number or expect a particular expression to be identical to when you did a waiver for it.

This has all been examined and the holes have been deemed unconcerning, yes ideally there'd be full waivers documenting this but as with any real life engineering project you can't do everything perfectly! There is internal documentation explaining the rationale for why the holes aren't a problem but it's not public.

> Getting waiver files reliably integrated into that flow has been problematic as those files are fragile, alter the RTL and they typically break as they refer to things by line number or expect a particular expression to be identical to when you did a waiver for it.

Yeah last time I did this we used regexes but I really don't like that solution. I think the waiver should go in the RTL itself. I don't know why nobody does that - it's standard practice in software. SV even supports attributes exactly for this sort of thing. The tools don't support it but you could make a tool to parse the files and convert it to TCL. I've done something like that using the Rust sv-parser crate before. Tedious but not impossible.

Also we found the formal waiver analysis tools to be very effective for waiving unreachable code, in case you aren't using those.

Congrats on the silicon anyway!

> Also we found the formal waiver analysis tools to be very effective for waiving unreachable code, in case you aren't using those.

Yes we had used them just never got it slickly integrated into the verification dashboard. We had used this kind of analysis for internal sign off. You could generate the waivers manually and check them in but that suffers from the problem discussed above. Plus as OpenTitan was a cross company project you run into EDA licensing issues where not everyone has access to the same set of tools and a UNR flow could be running fine on one partner's infrastructure but isn't workable everywhere for multitude of reasons.

The ideal would be the nightly regression would do the UNR flow to generate the waivers and apply them when generating coverage but as ever there's only so much engineering time to go around and always other priorities.

I think they're saying the coverage they have is to the highest industry standards, not that 90% is a high standard.

Reply on news.ycombinator.com