Show HN: RetroClip – Instant Replay for Your Mac | Hacker News Reader

Show HN: RetroClip – Instant Replay for Your Mac

realartists.com

309 pointskogir116 comments

Hi, I'm James and I wrote RetroClip. It's just a little side project, but I think it's neat and so I decided to release it.

I really enjoy technical how it's made blog posts, and I also enjoy elaborate desktop UI recreation web pages[1], so I made both for this project and I hope you like them. If you've got any comments or questions, I'll be pressing Cmd-R on this page.

[1] https://www.realartists.com/retroclip/

How much is the Pro upgrade? It's unclear from the website as it's an in-app purchase.

Also, this is a pretty amazing technical accomplishment, and an incredibly well-written blog post. Huge kudos.

5 bucks. :)

A fair price, nice work! :)

(or £4.99 in the UK, which is $7.10. But we're used to paying more! Also, bought :) )

Yeah, sorry. Apple decides how the price translates across different currencies.

Actually, that's £4.15, or $5.90, plus 20% VAT. But the GPB has been gaining value recently. 6 months ago it would have been more like $5.30 in the UK vs. $4.99 in the US.

I love the "elaborate desktop UI recreation web page", and it's inspiring me for my next Mac app's website. Do you have any resources to recommend?

I can't really recommend the approach in general. From accessibility or search engine optimization standpoints, what I've done is a disaster.

It's more just an objet d'art type thing, just this sort of stylized recursive, I dunno, joke thing. It was fun to make and for some reason Nick and I find the concept hilarious, but it's also ridiculous. When I told my family about it their response, to a person, was "but why?"

You're of course welcome to view source on the page and copy anything you like (it is just a web page after all), but if I was trying to market something serious, I would definitely do it differently.

What's really cool is that if you save a video, then play it, and then save a video again, the small overlay player will actually show up in the second video.

Did you have to take care of that yourself?

Yeah, I render the video into the canvas, as opposed to positioning a video element on top of it, so that the capture will work recursively.

This is hilarious. Great app by the way!

Wow. Not only did you develop RetroClip, but then you developed it again just so you could put up a live demo version in the browser. And it actually works!

That's some dedication...! (and definitely worth 5$)

EDIT: It seems like it doesn't work with DRM content. Or more accurately, Netflix refuses to work while it's running. Unfortunate, but expected (and not your fault).

I guess it's not really that surprising, but recording a 4K display eats about one and a half core on Core i5, 2.7 GHz.

Is this with the real Mac RetroClip? That sounds really high, like way higher than expected. Is it an older machine without a hardware video encoder?

Macbook Pro, 13-inch, 2015, Intel graphics 6100. So I guess not. The 4K display is connected via displayport, and runs downscaled at 2560x1440.

OK, I would expect that setup to perform really well (like just a couple % CPU use in activity monitory). If you have a chance, can you send a sample[1] to support at realartists.com? I'd love to see what's taking all that time.

[1] https://developer.apple.com/legacy/library/documentation/Dar...

Nice work!

I'm impressed by the CPU footprint. Shouldn't be a surprise as it's all hardware now but still great to see this which makes me comfortable let it running along.

Is there a way to change store path?

Also I changed hot key twice and seems both keys are in effect now.

RE: Store Path. Unfortunately, no. In order to write outside of the app sandbox container, an entitlement has to be approved by Apple, and they only would only approve ~/Movies/<AppName>/.

RE: Two Hot Keys. Ruh-Roh. Thanks for the bug report. I'll investigate. EDIT: Will be fixed in next update. :) As a quick workaround, just quit and relaunch RetroClip and the old hotkey will be unbound, leaving just your new one active.

> RE: Store Path. Unfortunately, no. In order to write outside of the app sandbox container, an entitlement has to be approved by Apple, and they only would only approve ~/Movies/<AppName>/.

In that case, a symlink would work nicely: https://tinyapps.org/blog/mac/200805230700_change_imovie_sav...

Did you consider making it available to a wider audience outside the app store, or do the economics just not work out? Curious to have an up-to-date analysis.

I mean, everyone has the Mac App Store at this point, right? I think the drawbacks of the Mac App Store are obviously numerous and well documented around the web, but I'm not sure that lack of ubiquity is one of them.

Can you not use Powerbox to allow the app to write to user-chosen folders outside of the app sandbox container? Or maybe that's the entitlement you mean and Apple wouldn't grant it? Anyway, cool app. Nice work and nice writeup!

https://developer.apple.com/library/content/documentation/Se...

Yeah, I guess I could use a security scoped bookmark. Might be worth adding.

Any way we will see an update which allows us to capture 'only' a specific window. ie recording terminal session for replay?

It's a good idea. I'll think about it.

I definitely would recommend using ScreenFlow for capturing screencast type videos where the content is at least a little bit pre-meditated. It will let you crop the screen after the fact, pan around, and do all sorts of neat edits. It's really well done and worth the money. It might fit your use case better. I'm not affiliated with it any way, just a happy user.

we actually have something in linux we use, which allows us to set a 'position' for recording (sets a border to record in) basically, and we mainly use it for confirming terminal output / quick runs of stuff while on screen shares, or in slack.

Its nothing thats 'ever' predetermined to be shown, its all stuff thats either on demand, or 'how did you do that' rather quickly.

being able to just hit a button, after i did the 'thing' would be super beneficial.

This is frickin cool man! Great work. Can’t wait to try it out. Thank you

How much of a speed boost did compositing the mouse cursor yourself get you? If it's big, I'd be really interested in hacking OBS to do this, so I can stream and use my computer at the same time.

I haven't tried OBS in a while, but a few years ago I was really into the Heroes of the Storm beta early on and I streamed it a little on Twitch.

I actually wrote my own streamer because I wasn't happy about the resources that OBS required. Some of the code from that streamer I ended up dusting off, repurposing, and improving to write RetroClip.

Now, it's been a while, so what I'm about to say may no longer apply to current versions of OBS, and my observations may have been wrong in the first place, but when I took a tour through the OBS codebase, I felt like it was maybe supporting too broad of a feature set and supporting too many platforms to be able to attain optimal performance. Like, I'm not trying to crap on it, as it's an incredibly impressive piece of software, I'm just saying that there seemed to be a lot of data copying, cross platform concerns, and extension points, and the pipeline for getting video in from the screen and then out to the network just had a lot of work along the way, work that could be avoided if your only goal was to run on macOS and send just the screen contents to twitch as fast as possible.

So I doubt that a micro-optimization like my cursor compositing hack is going to be of benefit to OBS. I did that to avoid doing any copies at all of the raw image data in code that I control, and I would guess getting to zero data copies in OBS would be a long journey. There's probably a lot of lower hanging fruit in terms of macOS specific optimizations that can be done first.

Just fired up OBS and it turns out it was defaulting to software encoding, and I switched it to hardware encoding and it's now quite a bit smoother.

Do you have your streamer program posted on GitHub or something? I want it.

Sorry, I don't. It would need some dusting off to to compile on recent macOS (it depends on OpenSSL which they removed I think in 10.11, so I'd have to switch to SecureTransport or something). It also doesn't do any of the things that I think are table stakes for twitch streams these days like animated interactive overlays, webcam control, etc, all it did was stream the screen and the audio (i did mux system and mic audio).

My understanding is that like 99% of the traffic on twitch is basically the top 0.1% of channels, and all of those streamers mostly just use dedicated streaming boxes or have super beefy PCs. The market for a bare bones feature set twitch streamer for people playing games on Mac laptops has got to be pretty small. Maybe it's just you and I. :) So, that's why I kinda lost interest in it.

Count me in.. It’s at least 3 people...

Damn cool demo page :-D

Hey there. So the app isn't really working for me. It shows the icon very quickly on the menubar and disappears. What's the best form to collect information about what could've happened?

Hi! I’m also working on a side project that has to capture screen data in real-time. Do you by chance know if there’s a way to exclude a window from the capture without resorting to the slow CGWindowListCreateImage API?

[1]: https://github.com/archagon/degausser

Sorry, I don’t. I mean, you could of course identify the unobscured area of the window and paint over it pretty quickly, but if you wanted to draw what’s behind that I don’t know of a way to do that efficiently.

Bummer! Just thought I'd ask.

Really neat project, by the way: I love seeing technically elegant solutions to these kinds of singular problems! Did you run into any Mac App Store sandboxing restrictions with respect to capturing the screen?

No sandbox problems to report :)

Dunno if I should be impressed or going “what the hell?” Seems kind of absurd that they enforce the sandbox pretty heavily in the MAS just to let things like that get by pretty easily. Apple confuses me.

I find the fact that you can do this in the DOM without special permissions kind of spooky! Does that mean that websites can be grabbing my screen if they are running in the background?

Thanks for your detailed writeup! I bought the app and am gonna use it!

There's a service called Smartlook (https://www.smartlook.com/) that does exactly this, at scale, for user analytics.

websites can't grab your screen (say, they can't grab what you have on your excel window, or what's your taskbar, etc), but they can grab what is inside their browser tab. Which is what his example is doing.

I think the Window Server hack they are doing could definitely be used for interesting exploits. He isn't just recording the DOM that's for sure.

This is incredible! Especially inspired by your success with WebAssembly.

In some version of reality, you run with this and go on to create a fantastically useful web-based desktop system and define a new category :)

I really enjoyed the blogpost!

> we knew we had to get with the times

There are already too many of us working on the web side of things! :p It's lovely to see someone crafting a fast native application instead of yet another electron app. Speaking of which, do you have any recommended literature for learning more about the things you're writing about?

How does this work with multiple monitors? Can I request the option for more control & longer record times? i.e. be able to set it any where between 1s-600s?

For context I downloaded OBS as much like you I was playing Fortnite BR and getting tired of having only static screenshots showing my win. OBS is ok but a performance hog. So your program looks awesome except 60 seconds is to short, sometimes the awesome play I want to share took longer than 60 seconds..

Also I play using an egpu on a second monitor so I only want to record that monitor playing. No reason to record the monitor showing discord.

Edit: I see you recommended quicktime for videos over a minute but as you've mentioned that takes up way too much diskspace to just leave running while you're playing. So I'm really hoping for some more options on the recording length.

It looks like you can configure it to pick one monitor or the other. It would be better for it to record all monitors, since the whole point of this app is to enable retroactive screen recording, and so I would like to choose the monitor after the recording is done.

I mean, I could do longer, it just takes more memory. 120s is probably doable though.

If I understand correctly increasing the duration just increases the amount of ram required right? How much does 60 seconds take on average?

Personally I'd like to set it to 5 mins which is what I have it set at for OBS's replay buffer. I find its much easier to trim a video than it is to merge multiple videos together.

I guess what I'm really asking is what are the real downsides to giving the user custom duration controls. So long as you provide the appropriate warnings / dialogs in a text box underneath the selections (OBS does that)? It seems like that would only increase the utility of product.

TLDR: Awesome piece of software that I think would be made even better if you gave the user more flexibility with custom durations.

Wow! This could be awesome for QA work. So often our QA team will manage to get our software into some weird state but not be entirely sure how they got there. If they used this they could hit the keyboard-shortcut to save the last 60 seconds as soon as they got to the error or weird state.

I've sent them the link!

That's a great use case and I could really use that right now.

Now if retro clip could also record keystrokes* and mouse clicks that could be even more helpful.

* OK, maybe keystrokes is a security issue? Not sure.

You could use something like xnee: https://www.gnu.org/software/xnee/

That says X11. Not going to work on macOS's Quartz.

Oh, you're right. I was thinking about linux for some reason.

That's a great idea. I wonder if there is similar software for Windows since that's where a lot of my companies QA issues are born (IE)

Nvidia and AMD both include similar instant replay software with their Windows driver packages.

They're geared towards gaming but I know NVs solution lets you enable desktop recording, not sure about AMD.

Ooooh this could be pretty fantastic for user debugging. Instead of "can you recreate that problem?" Just hit the keys and send the clip of what they were actually doing.

For a better integration the replay would show logs (for a desktop app: the logs of the client machine, for a web-app: HTTP logs) at the same time the video is playing.

E.g. a webmail app, if I click a particular email I get an exception, when the programmer who is debugging the problem is viewing the video, he would see what the server logged a few seconds after I caused an XmlHttp request through my click..

How about an option to make the Notification Center alert open the saved Finder location, rather than opening the movie in Quicktime?

Much easier to do something with the clip file, and you still have the easy (and perhaps better) option to watch it with QuickLook.

I'll think about it. In the meantime, I have a few pro tips that may be useful.

One trick is you can drag the icon from the QuickTime's window's title bar (or command click on it to reveal it in the Finder). Along with the ability to drag things while doing Cmd-Tab and Cmd-` that was added to macOS at some point, it makes it fairly easy to do drag and drop to other apps.

You can also use the Open Clips Folder menu item in the RetroClip menu.

Finally, you can also just disable the banner for RetroClip using the notification center system preferences and have it be stealth, just like screenshots are, and then just navigate to the ~/Movies/RetroClip folder in the Finder directly.

Fantastic work. I wish there was something similar for Linux. it sounds like you've made something that has minimal overhead.

I like the mouse-pointer hack, I guess you could take it further by only adding it when the user wants to save the clips (and it gives the option of having no mouse pointer, or maybe hiding it around the time when the mouse is not moving or being clicked).

I thought about that, but the problem is, I would have to decode and then re-encode the video on save to composite in the mouse cursor. As far as I know, there is no way to quickly and non-destructively composite an image on top of h264 video.

I should add, though, the idea of storing the mouse cursor separate from the captured video is a really good one in general. I am pretty sure this is what ScreenFlow does, because you can edit the mouse cursor in their app after a video has already been recorded. The tradeoff is that they need to spend more time and space during the recording phase to get really high quality source video, and then they have an additional export phase when you want to save the video out that does the rendering and lossy compression. That tradeoff is no problem for them, but doesn't fit RetroClip's use case.

Does it record audio as well? This, to me, is the biggest downfall to recording desktops on Macs.

It doesn't. For some reason, which only the CoreAudio team could explain, it is necessary to use a kernel extension to capture system audio on macOS. Kernel extensions are not allowed in the Mac App Store (and rightly so), so having it just work is unfortunately not possible.

That being said, I could easily add a feature to record any other input device on the system, and then you can install the Soundflower kernel extension (or one of Rogue Amoeba's other excellent utilities) to create a loopback audio input device.

I always watch short video game replays with the audio off or on gfycat or whatever where there is no audio anyway, and for capturing bug reports or whatever I don't need audio either, so I didn't feel like I was missing much without audio for myself.

All that said, if the CoreAudio team would come to their senses and provide a userland API for doing system audio capture, I would definitely be on that.

To add to that, I've been on the fence about adding audio muxing. I should just do it, and folks who have a Rogue Amoeba app installed would be able to make use of it. Compared to storing the video data, the audio data is tiny, so buffering a minute of it is no problem (even if I stored it uncompressed to save CPU cycles, it would be under 10MB).

One option is to just allow people to select a single audio source, that way you don't have to do any mixing and they could use a 3rd party audio capture tool as the input source.

Monosnap records video and audio, and is available on the App Store. I don't know offhand if they record off the built-in mic or if they hook the audio stream some other way, however.

https://itunes.apple.com/us/app/monosnap-screenshot-editor/i...

monosnap is pretty awesome ; very feature rich as well.

i used to use annotate but it got abandoned when dropbox updates their api so i switched to monosnap since.

http://blog.monosnap.com/how-to-record-video-with-the-system... - here the guide how to record system audio.

ScreenFlow does a fantastic job of recording the macOS desktop, including audio.

I've got ScreenFlow on Cmd-Shift-2 and now RetroClip on Cmd-Shift-5!

> I got the idea to write RetroClip after playing the game Fortnite Battle Royale and winning, and then having nothing to show for it besides a static screenshot.

You should try Shadowplay or the equivalent from your GPU vendor :) Great job on the project.

OP is a Mac user - Nvidia Shadowplay and AMD ReLive only work on Windows.

Can you expand a bit about your decision to ship this in the Mac App Store vs. outside? Especially if a non-App Store build could get you things like audio recording.

Our main product we do sell outside of the app store. It's a much bigger and more complex app, and I like not having to pay Apple a 30% cut on it, and I also like the ability to push updates whenever I need to without having to do app review.

That being said, what I like about the App Store, and I'm just getting my feet wet on this with RetroClip, is that I think it's easier for users to trust you on the App Store. The app sandbox gives people some confidence that an app is going to be well behaved, and if it isn't then Apple can always pull it from the store. That's not to say that our non-App Store stuff isn't well behaved, it absolutely is and we take it very seriously, but being in the App Store maybe makes it easier for users to have confidence in us.

I also think the in-app-purchase trial-ware model that a lot of developers (myself included) are starting to adopt on the Mac App Store makes a lot of sense to me. Even $5 is a lot to pay for software if you can't try it first, and I think that was a big problem early on with paid software in the Mac App Store.

If the target audience is gamers and QA-types of people, I don't think they will care where they get it from.

Really nice. What's the original intended use?

Thanks! My original intended use was to win at video games (or at least fail in amusing ways) and then post the replay clips to Reddit ;)

It's also useful for software development. If I had a nickel for every time I got a screenshot in a bug report but incomplete repro steps, I could probably buy a sandwich. So being able to retroactively get a video of what happened can be super useful.

Amazingly useful, I would say. Two use cases I hadn't thought of.

(Actually I just thought about jokes.)

This is very slick and easy to use. You sell it well by comparing it to the features on modern game consoles. I'm not sure when/if I'm going to use it, but I like the idea that it's there if I need it!

My only criticism is it adds yet another menu bar item, but I understand the reasons why that can be useful (and apps like Bartender mitigate this somewhat)

Thanks! I totally hear you about the menu bar. I used to laugh at Windows machines with all the crap in the system tray, and now look what I've become.

I'm on a Mac running 10.11 at the moment, so cannot test (10.12 or later is required), but are the only choices for capture length 30 and 60 seconds? Would be nice if this could be set by the user. Though for longer recordings, would RetroClip offer any advantage (performance, file size, etc) over QuickTime Player's built-in Screen Recording feature?

If you want to do a longer recording and you want to keep the whole thing (or substantially the whole thing), QuickTime Player is better. I also use, and love, ScreenFlow for doing screen cast type videos. The difference is, I'm not going to leave QuickTime Player or ScreenFlow recording all day because of disk space concerns.

As for 10.11, it probably would work if I just changed the target OS, but for a little side project I didn't feel like going through the trouble with testing 3 operating systems.

How about an option for 5s or 10s? Kind of funny the non-pro version can do 5s but the pro can't!

an alternative: http://recordit.co

What's the actual video format? I mean, my knowledge is a bit dated, but don't codecs like x264 usually have I B and P frames so you couldn't cut at an arbitrary frame without reencoding. Do you just emit I frames or does x264 have some more advanced tricks available?

It's not just I (key) frames. That would be really bad for quality or data rate. There's an I frame every 1 second, which gives me a place to cut. In general, not just for RetroClip, having I frames is important as it will allow for efficient scrubbing through the video in a playback application.

Ah right so you technically don't save exactly 60.0 seconds, but I guess that's OK ;)

Like I said my knowledge of this stuff mostly dates back to mpeg2 days and I could have sworn the I frames were further apart, except on scene cuts.

I actually pad it to avoid coming up short :)

This is pretty cool, but I'm seeing RetroClip take 300% CPU on my Late 2015 Retina 5K iMac. I thought it was maybe because I was watching a Twitch stream, but it still used 300% CPU when I paused it, when the only thing actually changing on the screen from frame to frame was a tiny animated GIF.

Wow, that's not expected. Can you grab a sample[1] for me? I'll see if I can find an iMac somewhere I can test on.

[1] https://developer.apple.com/legacy/library/documentation/Dar...

When I sampled it earlier it was doing a bunch of stuff in VideoToolbox. I’ll check again when I get home this evening.

Thanks. Sounds like we might have failed to open the hardware encoder and fell back to software encoding for some reason. I’m going to try to get some time on an iMac tomorrow and see if I can recreate the problem.

Ok I took a sample. https://gist.github.com/kballard/636fa3f6c09d6fc6e2a2a1efa33...

I also checked the Console. Here's what I found:

  default	19:55:42.767418 -0800	RetroClip	GVA info: preferred scaler idx 1
  default	19:55:42.769327 -0800	RetroClip	AVD encoder error: kAVDPictureEncode CreateContext, unsupported resolution 5120 x 2880
  default	19:55:42.808188 -0800	RetroClip	Using metal device <BronzeMtlDevice: 0x7f80bf037200>
                                                            name = AMD Radeon R9 M395X
  default	19:55:52.609993 -0800	RetroClip	LSExceptions shared instance invalidated for timeout.

My hero! Sounds like I should do some scaling for these 5k iMacs. I got away with up to 3840x2400 resolution just fine in my testing on various macbook pros, but I don't have any iMacs. I'm gonna go try one at the Apple Store tomorrow (not buy obviously, I'm way too cheap for that, but just use it for a minute).

EDIT: Yeah, I'm pretty sure 3840x2400 is the max supported by any recent Intel CPUs, and older ones are less. The iMac Pro might be a special case because I hear it uses its GPU for H.264 encoding. Anyway, querying what the max is for a given machine and then scaling to fit it was an easy fix. Hopefully I can get it deployed to the app store soon.

Glad to hear it! Not sure I need a 5120x2880 video anyway. I’d be perfectly happy with half that resolution.

I assume higher resolutions have a larger memory footprint. Is the RAM usage still reasonable at 3840x2400?

Finally just relaunched it again after the updates. it's now taking 10% CPU while watching a Twitch stream and 1–3% CPU when not. That's much better than 300% CPU :D

I'd love to purchase the Pro but I can't see the price anywhere. Nothing in the App Store or even when I click "upgrade to Pro" in the app itself no information on how much it's about to charge me.

It’s $5. I’m going to make it more clear in the next update.

Sweet thanks man. Love the simplicity

I always wanted something like this for video, rather than just audio:

http://plugin.org.uk/timemachine/

Thank you. I'll give it a try.

"Current generation video game consoles all have a feature where you can press a button and capture the last minute or so of gameplay and I wanted this for my Mac."

PS4 can record up to an hour, fyi.

can't wait for the Mr. Robot episode where RetroClip is mentioned and the hackers use the last 60 seconds of someone's screen without their knowledge.

Well, it's totally sandboxed, no network connections are allowed either in or out, and the video is only stored in memory, not on disk until you press the keyboard shortcut, so it should be pretty secure.

This is actually a feature in the Nintendo Switch too. It will save the last 30 seconds of footage prior to you invoking the capture input.

Yeah, my 4 year old son loves to spam the capture button. We have soo many Mario Kart clips.

Cool. I actually wrote a golang project to do this on OSX. And then convert it to a gif.

Is there anything similar for Windows?

Yes! Check out the Xbox app for Windows 10. GPU drivers also provide a similar feature (nvidia’s is called shadowplay).

shadowplay by nvidia.

Are the streams or mp4 files being uploaded anywhere? Any transparency you can offer about this not potentially causing privacy issues?

I run little snitch, the application doesn't appear to make any outbound connections

It's totally sandboxed. It can't make network connections, in or out at all. You can verify this yourself like so:

  $ codesign -d --entitlements - /Applications/RetroClip.app
  Executable=/Applications/RetroClip.app/Contents/MacOS/RetroClip
  ??qq?<?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
    <key>com.apple.application-identifier</key>
    <string>EL9BUK3ZCV.com.realartists.RetroClip</string>
    <key>com.apple.developer.team-identifier</key>
    <string>EL9BUK3ZCV</string>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.assets.movies.read-write</key>
    <true/>
  </dict>
  </plist>

Very cool idea

Reply on news.ycombinator.com