creationix

What do you mean by common.js implementation? The only thing I know of that survived the common.js movement was that node.js adopted it's module file format. That format requires either sync I/O, XHR loading and wrapping or a build-step or build-server to use in the browser. There are libraries out there that try to solve the module problem in the browser, but there are certainly no clear winners and nothing that doesn't have serious limitations or drawbacks in one way or another.

Also as I mentioned and you quoted, I found a great way to load common.js modules in chrome apps that nobody had done before, but it only really works in that unique environment.

But modules are just the tip of the iceberg. I need a way to handle I/O actions, streams, TCP, fs, db, etc. All these interactions and APIs need to work cross-platform in a variety of different runtimes. I can't even start implementing git clone over a TCP socket till I have a TCP socket API to program against.

But like I said in the bountysource, most of that is all now solved more or less. I have a much better understanding of what needs to be done and I am starting out this phase knowing how git works internally down to even the huffman encoding bits inside the deflate inside the objects inside the packfile stream inside the git clone stream.

I also have more time this round and less conferences that I have committed to speak at or visit. (I think I only have two more trips the rest of the year actually)

I'm sorry the progress seems to disorganized. I've recently moved to doing work in the main js-git repo to make things more obvious to people.

As far as the first fund-raiser, I was able to stretch the money farther than I had anticipated but as I mentioned in the second fund-raiser, has a slow start.

It's hard to build a large library like this when basic things like streams, packages, module loading, and binary data are so varied and there are no accepted standard ways to work. This is pioneering work, not paving cowpaths.

This is the second time they've backed my js-git project. They were the largest backer in the original kickstarter. I asked them back then why they backed my project. The response was that they believe in open source software and like seeing cool things being written in JavaScript.

As far as I know, there are no Mozilla projects that benefit directly from js-git and they their goal is to promote open source JavaScript in general.

Yes, I will be making sure my code runs on FirefoxOS as well as ChromeOS, Internet Explorer 10+, iOS, Android, node.js, etc. (basically anything that has network access, a binary data type, and some form of storage).

Correct. Also the amount of code generated by emscripten is too much for me. I want js-git to be a light-weight dependency.

There are many other ways to store data locally that vary by platform. They range from full filesystem access in node.js to html5 fs apis in modern browsers and chrome apps. Winfs has it's own APIs ported over from C#, Firefox os has fs. Also there is IndexDB in most browsers. LocalStorage is probably the only thing I'm not going to try to use.

There are several failed attempts. Most of them died because they were just side-projects and/or programmed in a monolithic manner and crumbled under their own weight. The two main devs on js-git are myself and Chris Dickinson. I'm working on it full time so it can have the attention it needs (hence the fundraiser) and Chris has a couple of failed attempts in the past and knows what bad areas to avoid.

There is nothing wrong with JavaScript itself that makes this infeasible.

I'm implementing git in JavaScript to bring more independence to developers on more devices. This will enable more apps to have distributed data that works fully offline and doesn't require cloud services to work.

One benefit of this as mentioned in the fundraiser text is to build developer environments for kids.

But another benefit that's less obvious and very relevant to this week's news is it moves power back into the hands of the users of software. Much of software has been moving to the cloud in recent years and most new devices can only work with cloud data. You can't even log into a chromebook, surface, android tablet, or iPad without registering with some online service and giving them your personal details.

If you want to write some software on these devices you have to use the device like a thin-client and be at the mercy of your internet connection and some cloud service (like online IDEs). The platforms, for the most part, don't allow much local work to be done.

They do all allow JavaScript and have some form of network connection and persistent storage. By implementing git in pure javascript we'll be able to build tools that work completely offline, peer-to-peer or, if desired, with cloud based systems. I believe the owner of the hardware and software, the author of the code, should decide how and where it gets distributed.

But I need funding to complete this project. I have a house, a wife and 3 kids to take care of. I've quit my paying work so that I can make time to finish js-git. It's getting close, but I need just a little more time to finish it to be something useful.

If you care about giving control back to the user or believe in enabling kids and think this is a useful way to accomplish this, please back me and/or spread the word.

But that's exactly my point. Where I live in the poor south, most kids have iPhones because they are "free" and their parents can't do the math to realize that the monthly fees make it incredibly expensive. But I have yet to see a single local kid who owns or even has access to a mac laptop. (not to mention the $100 for the developer license is a month's rent for some of them.) So as far as they are concerned, it's impossible to develop on their phone.

I can only judge on what I've seen personally and what I've heard from industry leaders and both line up.

Wherever I'm living I go around to schools and groups looking for kids to teach programming to. They all have computers of some sort (mostly iPhones and some tablets), but don't know how in the world they can develop on them. The only way to write native apps for iOS is to buy an apple laptop, get a developer license and learn an advanced programming language. That's beyond the reach of most kids.

Now suppose that new websites popped up using my library and all the kid needed was a device with a modern browser and optionally a bluetooth keyboard? They could learn JavaScript and write webapps. There is huge demand for new programmers in our economy and most kids only see electronics as consumption devices.

Speaking of, my Chromebook just arrived via UPS, now for some fun!

Thanks for the links. You are correct that I probably won't be able to support all git repos because of memory constraints and unless I get special APIs from the browser I won't be able to do git protocol or git over ssh. I do feel pretty confident that I can implement dumb and smart HTTP though.

Even if this library can't be used for everything, it will allow a whole lot more use cases that are currently possible.

I will explore this as one of the possibilities. I'm not dead set of reimplementing everything myself. If using an existing JS library and finishing it or compiling an existing C library using emscripten is the best, then I'll do that..

The point is I'll spend the proper time to evaluate all the options and fill in any gaps needed to have a solid solution.

Thanks!

Not all web apps require constant server interaction to develop. Also you could probably mock a lot of the server interaction as well.

Like I said, my main use case is mine. I want to develop on my new ChromeBook Pixel (if it ever gets here) without being forced to use an Ubuntu chroot. Packaged apps have much larger size quotas and most my code repos are small anyway.

As far as the education aspect, I have future plans to create a new programming language aimed at learning to program that compiles to javascript. Combined with a proper in-browser IDE, this would allow any kid to learn to program on any machine that has a browser.

It obviously won't replace native dev tools, but it will make web-based stuff a whole lot more powerful and that's something I and many others feel is worth paying for.

So I can develop offline using nothing more than a browser. I clone a repo from github to my browser when I'm online. Then I'm offline in a bus or on a plane, but I can still work because browsers have file storage now. When I'm done working, I can go online and push my changes back to github.

And all this will work anywhere that has a browser, ChromeBook, Surface RT, iPad, etc..

Obviously, it's mostly of use to web developers since JS is the only runtime, but I also have other later projects attacking that problem. But for now, JS only is a pretty big market.

Exactly the kind of person I'm trying to help!!! Thanks for the support.

Thanks for the feedback, I've improved the copy on the kickstarter page.

No, I'm just writing a library in this phase. I'll have a prototype that uses it, but I expect the IDE authors to want to integrate it themselves.

Thanks Matthew, that's the plan. Also since the target audience is developers, I don't have to support ancient browsers. This isn't a news site for the general public, it's a developer library.

I want this, I know many people who want this. That's good enough in my book to create it. If enough people believe in the idea enough to fund me, then it's a good idea.

I know this is a hard project, that's why I haven't done it yet in my free time. I realize that this will require my full-time attention. But click through the links of my other projects in the kickstarter. I'm not new to hard problems and low-level coding in JavaScript.

I don't have a detailed roadmap and I can't make performance guarantees because I don't know yet. But I do know that I'll do whatever is possible within the constraints of a web browser. Again, I'm not new to these constraints. I believe the web is finally mature enough to pull something like this off.

The main problem it solves is to allow offline development in sandboxed environments like ChromeBooks and Windows RT devices (or even iPads if you had a BT keyboard).

The main thing it allows is developing offline en environments like chromebooks (like the new Pixel). I worked on Cloud9IDE for a year and it was a great experience as long as you were online with a fast connection. With this library, html5 apps will finally be able to do the full developer lifecycle. They can clone from github to the browser's local file storage when online, work offline using an editor like ace or codemirror, and then when they are online again, they can push their changes back to github. I'll implement branching, merging, diffing, and as many other awesome common tasks from git as possible.

I've wanted this for years and I'm super excited to finally be able to create it when this kickstarter finishes!

I never used the "official" js codec because it didn't exist when I started. My codec has been optimized for nodejs and does rather well there. I recently did a jsperf for the browser port (msgpack-js-vs-json) and while slower than the nodejs version, it looks a bit faster than the "official" one.

Interesting format. It appears easier to implement than msgpack in a scripting language. My gut feeling is that mspack will be slightly more compact and faster to decode (especially if decoded using C).

That makes much more sense, thanks.

Interestingly, in Firefox, the gap is smaller, their typed array implementation is a bit faster. For the number heavy case, msgpack is only 47% slower which is close enough in performance for a great many use cases.

Just an update, it turns out that Typed Arrays aren't as fast as node's Buffer implementation. When comparing msgpack-js-browser to the native JSON library, JSON is way faster in chrome. http://jsperf.com/msgpack-js-vs-json

However, in the number and array heavy case, the msgpack is 2.5x smaller when serialized. So even if it's a bit slower in browsers, the bandwidth savings and the ability to store binary data may still be worth it. (remember that performance in the browser scales very differently since it's distributed across all your client's browsers)

It's not a new format, I have no idea why it's popular news today. Having worked extensively with both msgpack and JSON in a javascript environment, I can tell you, its the closest to JSON of all the binary formats. The difference with JSON is msgpack's strings are binary safe (you can have a png as your value) and the format is a bit more compact, especially around integers.

JSON has: numbers, strings, booleans, null, arrays, objects. Msgpack has: numbers, raws, booleans, nil, arrays, maps.

So I guess msgpack is a superset of JSON. The raws can contain utf8 encoded strings like JSON mandates, or they can contain other things. There is no technical reason that the keys of the maps have to be strings. You could take a lua table that has another table as key and encode that in msgpack just fine.

In practice, I wanted more out of msgpack, so I extended the format using some of the reserved byte ranges to add in an "undefined" type and a distinction between utf8 encoded string and raw binary buffer.

For me this new format has been extremely useful as a general data serialization between processes (node to node, server to browser, etc..) I usually use it over binary websockets or raw tcp sockets.

Is was a fun project actually. https://github.com/creationix/msgpack-js-browser

That benchmark you referenced has nothing but strings for it's data. That's not msgpack's strong point. Do another benchmark with nothing but small integers and arrays and you'll find msgpack kicks json's tail in both cpu usage and bandwidth, even when comparing the native JSON.parse with my pure javascript msgpack implementation using typed arrays.