Mozilla launches voice search, file-sharing and note-taking tools for Firefox

"this can be very easily done fully client-side" : maybe, if you have the voice model and and inference engine that runs well on devices. Mozilla doesn't have that yet, so this experiment uses a backend running a Kaldi server and model that uses too much memory to run locally.

Once DeepSpeech is ready I'm pretty sure they will switch to that, and ultimately to on-device voice recognition with PipSqueak (PipSqueak is expected to be an inference engine usable on devices). Unfortunately none of these projects are far along enough to be usable.

Common Voice is mostly related to DeepSpeech as this will help getting data to train the engine.

Here is the backend: https://github.com/mozilla/speech-proxy (which talks to Kaldi via I think https://github.com/api-ai/asr-server)

> Actually, I think this can be very easily done fully client-side, with good accuracy. Even on Android, the voice recognition can run client-side / offline.

I'm not sure I'd say it's easy; you will certainly trade off accuracy versus a state-of-the-art server model. Among other things, Firefox users are not going to download gigabytes of recognition model, so it'd have to be a lot smaller than the server ones would be.

Very possibly it will be slower too, since the servers would most likely be using GPUs for at least parts of the recognition, but it might not be easy to ensure the same on all the millions of PCs Firefox runs on.

It's interesting if Mozilla is running their own speech recognition system. I wonder whether it would actually be usable in practice... The problem is, I couldn't find any kind of online demo.

> If I wanted voice search, file sharing and/or note taking I would find the appropriate piece of software for that task and install it separately.

The good thing is that these are all separate browser extensions, none of which are bundled by default. So that's exactly the case.

The title is misleading. These are not part of Firefox. They are in the Test Pilot program, which is a way for Mozilla to gather feedback on what features they should include, how they should change them if they do include them and all in all is completely open-ended. They might also release these as extensions or somesuch. They are all coded as WebExtensions (except for that file-sharing thing, that's just a webpage), so that's very well a possibility.

Also, browsers already have code to record from your microphone, due to the WebRTC web standard, so it's not like this opens an entire new class of potential vulnerabilities.

I wonder which browser you're using, because there are plenty that come rather close to your definition (e.g. Midori), but odds are really high that you're not using them - because nobody is.

Thing is, if Mozilla wants to stay relevant, it has to find out what users want and deliver that. These experiments are one way to do that.

> I just wish it was .. faster.

They're working on it, to see the progress, install nightly (which is already a huge improvement) and turn on servo CSS for an extra boost. It feels much faster.

Personally I've found the recent Nightly with the new Rust CSS engine to be pretty snappy. I'd say anecdotally faster than Chrome.

There was also a recent post regarding having large numbers of tabs open in Firefox (see https://news.ycombinator.com/item?id=14823807) and it did pretty well.

The auto playing issue is fixed and making its way down towards general release.

It doesn't help that they keep cramming features into Firefox. Chrome feels pretty bare in comparison.

I have Chrome as reference to my FF here and since their beginning, it never was faster again. As in every FF post, there will be people saying that and the opposite. I guess the difference depends on other factors like Addons or Plugins as well as the probably narrow difference between both.

I never really came upon a reason to leave FF behind. It works good, has all the addons, and I can still make it look like a real window with options and things where they should be. I also have a "special" relationship to Chrome due to this "bundling to freeware" they use to push the browser onto people who don't want it.

@your tab problem: I just googled this up https://addons.mozilla.org/de/firefox/addon/load-tab-on-sele... No idea how good it is though since I never felt I'd need that.

There is a solution/addon for everything ;)

I always admired Mozilla for their efforts in making Firefox better. But I switched to Chrome many years ago. For me, the problem with FF was UI and UX.

Chrome and Safari just feel more "smooth and sleek", and UI elements are consistent (Look how Chrome buttons are rounded-rectangles).

The other reason I switched from FF was that Chrome has always been simply better. Sure, FF can handle many tabs, but I save that for times I want to work with Selenium or something. For an ordinary user (and even pros), Chrome just beats FF.

Weirdly enough - I found gmail was running VERY slow on Chrome. I switched to the Brave browser and it's been blazing fast.

Weird that Google products don't even seem to run the best on their own products.

Firefox stable feels relatively slow. Firefox nightly is really snappy, especially with Stylo enabled.

Also, go to about:config, and make sure preloading is turned on. I checked it last week and for some reason it was turned off, even though I don't recall ever doing that.

It's getting there. They recently added multiprocess support. A new CSS engine is coming. In the further future we're going to get more and more pieces of their new rendering engine in.

Personally, I dropped Chrome 2 years ago or so, for the same reasons (moving away from everything Google that I reasonably can). Firefox is more painful to use, for sure, but it's gotten "okay" enough that I'm willing to keep using it in favor of its benefits (privacy).

By the way: In Firefox, open `about:support` and check that "Multiprocess Windows" says something like "2/2 (enabled by Default", where the numbers can really be anything but 0. If it's disabled, that means your Firefox isn't using the new multiprocess support, most likely because you're running an incompatible addon.

Google does not want you to move from their services and APIs.

Chrome/Chromium, as a recent example allowed serviceURI in Web Speech APIs for third party recognition to be plugged in. Here: https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecog...

It was dropped in Chrome 49 and we're now all stuck with using Google. There has been little coverage on this, some speculated it was hardly implemented, or was dropped because lack of a standard API format.

Whatever the case. Google / Chrome choose not to fully develop that feature and its now gone. Which ultimately works in Googles favor.

Option to disable autoplay in background tabs exists in Firefox for so long that I forgot it isn't enabled by default.

Go to about:config and set: media.block-autoplay-until-in-foreground = true

In older versions they used another config variable media.block-play-until-visible .

There is also media.suspend-bkgnd-video.enabled to stop decoding video in background tabs after set amount of time (10s default).

Pale Moon, been using it for a few years OR Chromium.

This has a lot to do with the ever increasing incompetence of web devs than it has to do with Firefox imho. I do performance testing for a living and Firefox creams Chrome on sites where the devs know what they are doing. I don't think of Chrome as faster, but better designed to hide developer mistakes. Chrome basically keeps the mediocre developer who can't rtfm in business just the same way Microsoft did back in the day. Same pointlessness on Android. I call it the Trump definition of success - pander to the lowest common denominator and pretend there isn't a price to be paid.

When did you try?

I've been on Firefox developer edition for awhile now, because it's just SO MUCH FASTER than chrome, especially on high tab volume.

What does "faster" mean? honest question. I've been using Firefox since its early days and never jumped on the Chrome bandwagon simply because I didn't see the need to do so. When Firefox felt slow, it usually was the whole operating system that's slow and neither Chrome or any other browser could solve that. Nowadays, all the machines I use are powerful enough to not feel any such slowness and I just don't understand what is this "speed" people keep attributing to Chrome and what is so "slow" in Firefox (regardless of that fact they're working on speeding it up).

In addition, Chrome recently crashed on me multiple times while using Google Spreadsheet which is supposed to work better with Chrome than other browsers.

~as an asside

Its not a real solution, but for twitch, which i agree is essentially unusable, ive replaced it with streamlink + mpv + a couple of scripts to notify me of who i follow is live. Twitch chat works in irc too.

Try disabling HTTPS Everywhere, if you have it. It made everything many seconds slower for me.

Speaking of speed, or lack thereof on Googke properties, has anyone else noticed how insanely slow the Google Admin interface is?

Oh c'mon it's not that much slower. Some things feel chunkier (scrolling) but JS performance is 80%-90% of the way there.

I find FF actually faster than Chrome. But maybe that's because I actually open multiple tabs which load in background, then I just switch between them. Though to be fair, I never knew about this difference... It just felt that Chrome is slower somehow.

I wonder if they are both the same, it's just that us users have learned different workflows based on strengths of each browser?

I think part of the issue here is that it just visually seems slower. In my experience it isn't actually doing things much slower, it's just a little visually clunky in comparison. And as far as resource usage goes, RAM is way down now because their multiprocess implementation is a lot less intensive than Chrome's. I switched for that reason and I'm enjoying it a lot.

Quoting from the article:

"These are obviously experimental tools and there’s no guarantee they will ever make it into a release version of Firefox. Indeed, the idea behind Test Pilot is to allow the Firefox team to test new concepts."

The efforts on making Firefox faster and more responsive have been going on for a few years (e10s/electrolysis being one of them) and are appearing in the releases as fast as the team can develop, test and deliver them. I personally believe Mozilla has a highly capable core team on this, and to insinuate otherwise, to me, seems like accepting commercial interests of other browser makers as higher than FLOSS interests. I don't disagree on quality of the product being a huge aspect in value judgments, but it cannot and should not be the only one.

Exactly.

I've been waiting for working and full printing support something like https://www.smashingmagazine.com/2015/01/designing-for-print... for years now. Printing articles from the browser is the same pain as 10 years ago.

But yes this is not as fancy as voice search or file sharing (or their single sign on, identity and all the other failed stuff).

Mozilla has been a vehicle for people who play out their tech and political dreams and then move on. And FF is crumbling all the way.

But this misery is soon over, we're in the end game of Firefox.

https://www.w3counter.com/globalstats.php

(Written from FF - a browser I've used since Mosaic)

Seriously. They're behind in speed, security, quality of the developer tools etc. and introduce new useless toy features and one re-design after the other.

I appreciate a lot of stuff Mozilla do outside the browser for the open internet, but the development of Firefox is pretty bad and it looks like the project leaders are incompetent.

Firefox is a sad example of how a successful software can be overtaken by a new competitor within only a few years due to a lack of focus.

That's a weird take. Mozilla has a wide variety of developers, working on all sorts of things. One of those things is experiments to figure out what the browser "foundation" should actually include.

I like that they're experimenting to figure out what good usability looks like and what the next generation of "browser" should deliver. Some things should just be there when you want them...not an app you install separately or a website you go to for the functionality. I don't know if all of these new things fit that category (voice stuff probably yes, notes probably yes, large file sharing maybe?), but I know I want Mozilla to keep trying new things.

Also, if Mozillians aren't building user-facing things with the "foundation", they don't actually know what the foundation should look like for developers. You have to be a developer using the platform daily in order to understand what it does right and wrong. I don't know how they could do that without actually building things that sit atop the platform.

A lot of companies built the platform, and populate it with some of their own apps to set a minimum bar of quality, like with Apple (Reminders) or Google (Keep). One might say the same discussion applies to language creators -- bake it into the language, or have 3rd parties do it? Not always an obvious discussion, and so perhaps the statements about ignorance and arrogance come too quickly.

Some things are too important to not at least put out a standard-bearer.

After the 54 release, it is the better browser frankly, low latency, and not a resource hog. Plugins, extensions, privacy and security defaults all seem to be better as well.

Using firefox focus on android is pretty great too. The normal firefox on android is just slow, not sure why.

It won't happen unless Google decides to abandon Chrome.

Mozilla really needs to outdo themselves just to regain the market share they lost to Chrome. Compared to IE, Firefox was clearly the better browser, but it took many years just to become significant.

I use VimFx on Firefox, I guess that scrolling issue doesn't bother me because I hadn't noticed it until I visited the link you shared.

I've started looking for VimFx alternatives since the following doesn't sound great:

https://github.com/akhodakivskiy/VimFx/issues/860

https://github.com/akhodakivskiy/VimFx/issues/860#issuecomme...

I'm still using Pentadactyl, which is incredible, but it's almost certainly doomed. You'll need to use the signed version on github: https://github.com/willsALMANJ/pentadactyl-signed You might consider Vimperator, which is similar, but inferior iny opinion. It has better support, though, and is more likely to survive into the future.

If you're looking for something simple, VimFX works, but it's too basic for me. Not sure of the future of VimFX.

Vimium-FF <https://addons.mozilla.org/en-US/firefox/addon/vimium-ff/> is the only Vim-keyboard-binding plugin I know that will work with Firefox 57 and above. I haven't noticed any scroll jankiness as you describe, but it's possible I have my key-repeat-delay set low enough that it doesn't affect me.

Can't say it has ever bothered me, but the future of Vim extensions on Firefox once XUL in extensions is phased out for good is dire. Right now, there is a port of Vimium (which I suspect gets my FF 55 to hang). I don't think any of the other extensions are likely to make it.

I use keysnail for the emacs key bindings but I think it has a vim mode too

Isn't Electron based on Chromium? Also, why would they be worried about it?

Chrome cannot control, for example VS Code.

Speed is definitely an issue on the mobile app. Personally I think Mozilla should try harder with the mobile market. Providing an ad and tracking blocking browser is still desirable on Android. Firefox is just too slow at the moment.

I really hope Mozilla take Servo and prop it up as an Electron competitor, as it might actually make a chance there and its incomplete compatibility with hacks used in the old days of the web are not that much of a problem there.

On the main https://testpilot.firefox.com/ page, scroll down and click "view past experiments". Each one gets a write-up when it's over.

Tab Center Redux is the official successor to Tab Center: https://addons.mozilla.org/en-US/firefox/addon/tab-center-re...

It's now a WebExtension, which means it will work in Firefox 57+ (and possibly eventually Chrome, if they end up supporting the same APIs), and uses the new Sidebar APIs that the new Notes experiment also makes use of.

If that doesn't fit your use cases, there are also quite a few new tab sidebar WebExtensions popping up that aim to replicate existing more sophisticated tab management extensions like Tree Styles Tabs (https://addons.mozilla.org/en-US/firefox/addon/tree-tabs/), and others that try to innovate on their own terms, such as Sea Containers (https://addons.mozilla.org/en-GB/firefox/addon/sea-container...), which makes use of the new Containers experiment.

Overall there is a huge amount of very promising new development in the WebExtensions space, and this is exactly the kind of innovation I had hoped WebExtensions would encourage.

If with experimental home page you're referring to Activity Stream [1], something similar cropped up in Firefox Nightly around a week ago.

Looks like this: http://i.imgur.com/nhGpHYt.png

Each section can be hidden individually, so if you just want the search box or just the websites you frequently visit or don't want stupid articles recommended on your newtab-page, those are all things you can do.

Seems like they took out the recent activity thing, but no idea if this is finalized already.

[1]: https://testpilot.firefox.com/experiments/activity-stream

There's many such tools available, RiseUp runs one called Up1. See https://share.riseup.net/

It's quite nice, has integrated image and video viewers as well as a pastebin all end-to-end encrypted.

>What's Send encrypting with? It doesn't show anything related to setting up a password during the video.

Likely a random string that is added to the download URL as Fragment (the part after # that doesn't get sent to the server).

The download URLs look like this:

https://send.firefox.com/download/50d43ef5f3/#GqCJXOUnCyxYGc...

I agree that there is very little explanation beyond "look at this cool thing".

3. Kaldi

2. Notes source code is at https://github.com/mozilla/notes, the panel.js source[1] is the one that saves and load content. It uses the Storage API from Web Extensions[2], but it uses it only to save/load locally. It is not currently sending any data anywhere. So, sync is not enabled. If you click the sync button, you're greeted with a message[3] saying it is not implemented yet (they do count it and metrify it, probably gauging interest).

[1]: https://github.com/mozilla/notes/blob/6bae20ceb58c5f1487885f...

[2]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/AP...

[3]: http://take.ms/ieu4n

Look at Hypothesis, which does something better : it allows you to annotate your web pages.

They've launched these specifically to gather feedback, so do put that into the feedback form. These are in no way finalized and it's definitely possible that they'll take that into account.

I believe the initial inspiration for this was people asking for a place to quickly jot things down that they'll then copy over into other applications, so might be a bit awkward to get those two use-cases under one hood, but I'm sure they'll try, if the demand exists.

Doesn't Edge have this built in?

Are Mozilla paying for people to do these experiments or is it people sending patches? I don't know but I'd imagine a lot of it is the latter.

Have you tried a recent Nightly? Firefox is getting faster: http://www.techradar.com/news/firefoxs-blazing-speed-with-hu... [1]. As Mozilla continues to pull in code from Servo and implement e10s, Fiefox will perform better and crash less.

[1] discussion: https://news.ycombinator.com/item?id=14848836

Try developer edition or beta to get a taste of the work they've put into speed. I ditched chrome for FF and haven't looked back because even a year ago, developer edition was so much faster... Especially on heavy tab loads.

Iirc the speed changes can't start landing in stable until November, because of the shift to Webextensions.

Agreed. This plus a number of plugins disappearing in FF57 with the new WebExtensions have forced me to switch to Safari.

On the plus side now, I get my tabs across all my devices.

These features are only in the Pikot program...a Mozilla extension.

No, not at all. These are in the Test Pilot program, which is specifically for trying new things and getting feedback if and how they should be included in Firefox.

So, most of these are not here to stay for the long term.

They might be included, they might be published as an extension either officially or just forked by someone, but overall, no guarantees on anything.

These Test Pilot experiments usually last for a few months, but there's no fixed end date, I don't think. It's mostly just a matter of having gathered enough feedback.

But well, if it seems like a feature that people want, then it gets included into Firefox itself and stays for longer.

Then, again, I for example can't at all imagine them including that file-sending mechanism. I doubt they even have the server capacity for that...

Lots of people are faster at speaking than typing. Or maybe you're reading from a document (or pill bottle) and don't want to type with one hand or whatever. But what I usually use it for is if I'm having a conversation with someone and we want to look something up. It feels more anti-social to type silently for a while than to make the computer part of the conversation.

"Firefox! I've fallen and I can't get up!"

There are many use cases for this, among them:

* Mobile version of Firefox. It might be easier to talk to fill a form on mobile than type on glass screen.

* Accessibility. There is a huge under served demography of people who will welcome this as typing might be hard for them.

You may not be on any of the groups above but that doesn't invalidate the features.

Actually, people who are very deep into the "get off my lawn" phase of their life, that is old people, are generally not the greatest typists. For them, the ability to talk instead of type, everywhere and not just in google search (which already has this) seems like a huge help.

I'll never use it. It's completely useless to me. But if my grandma actually did anything else than Facebook on her computer I'd show this to her.

Just to clarify these features are not yet on the release.

Firefox is as fast and lighter, and has been for a while (the exception being Googles own webapps that are heavily Chrome optimized). This is stuff they are testing out in TestPilot, so it's for people who want to go add the TestPilot add on and then selectively enable these features.

So your complaint really makes zero sense.

> Chrome has been and continues to eat FF's marketshare alive because it has been both faster and lighter.

I believe Chrome got a larger market share also because it had the giant force of Google marketing it heavily on its properties for years, whereas Firefox only had some support of that sort from the Yahoo search deal (IIRC). More people land on Google's online properties than on Yahoo's.

I personally don't agree that Chrome is faster and lighter. For me (as a heavy multi-tab user) it has always been slow as molasses and a laggy browser.

What's stopping you? Optional experimental plug ins?

They will have to adapt. For now they're using data from the experiments to guide API development. For example, the Tab Center experiment used old APIs, but now they've expended the new API and are building a new Tab Center Redux extension at the same time. https://addons.mozilla.org/en-US/firefox/addon/tab-center-re...

We're moving to a webExtensiony future. The idea here is that we'll be a first party consumer of the webExtension APIs so we can help test/drive/expand capabilities.

Note that Voice Fill is actually a WebExtension : https://github.com/mozilla/speaktome

can't you install umatrix/ublockO in mobile FF?

Go into about:config and set "privacy.trackingprotection.enabled" to true. This is default-enabled in Private Browsing.

Also, uBlock Origin and similar are perfectly functional extensions. Takes just three clicks to install uBlock Origin on a fresh desktop Firefox installation.

If you're wanting to tell me that this content blocker should be default-enabled, good luck explaining to webpage owners that they won't make ad money off of Firefox anymore, but should still support it.

This is the only one I can think of, not sure why that site would require it though. https://caniuse.com/#feat=input-file-directory

These featured come in an optional extension and your comment is fud.

They actually did, the recent switch to multiprocessing made a massive difference. I've been happily using Vivaldi for a few months now, but I'm tempted to go back to FF as the speed gap has been wiped out of late - only cold startups are a bit slower, actual pageload is absolutely on par or better. This on Mac, at least, but I expect it will have been similar on other platforms.