214 karma · joined May 7, 2010
Current startup: Matterbeam - rethinking data
site: https://matterbeam.com
blog: https://mikepk.com
email: mikepk (@) matterbeam.com
former startups: Smarterer Grazr
I think it was funnier in my head than on the page (and maybe I've thought about data architectures for too long).
Disclosure - used AI for brainstorming and some of the phrases (and the death star image).
The net effect was you could make your own news feeds / timelines and use code to control how they were filtered / combined / etc... It was crazy powerful (for 2005) and I still miss it _today_ since it had the dynamism of the news feed, some of the social aspect, and total control since there was no algorithm other than your and the people's who's list you subscribed to curation and any code you ran against it.
Not a lot left from those long ago days but I did find one slightly-cringy video: https://www.youtube.com/watch?v=45DSrU23sPI
:)
https://techcrunch.com/2006/09/18/grazr-10-blasts-off-into-t...
GameScape (https://gamescape.app)
Basically I wanted something to approximate pen, grid paper and physical tokens during the pandemic when I was playing with some old high school friends remotely. Roll20 was just too clunky.
I’ve got a new startup now (unrelated) so I haven’t had much time to work on it. If anyone wants to work on TTRPG tools (I’ve considered OSS this, or maybe even paying for dev) let me know!
This post doesn't go far enough. It challenges the assumption that everyone's data is "big data" or that every company's data will eventually grow to be big data. I agree that "big data" was the wrong model. We also need to challenge that all data should be stored in one place (warehouse, lake, lakehouse). We need to challenge that one tool can be used for every data need. We need to challenge how we build systems both from a technology and people standpoint. We need to embrace that the problems and needs of companies _are always changing_.
We are living with conceptual inertia. Many of our patterns are an evolution from the 70's and 80's and the first relational databases. It's time to rethink how we "do data" from first principles.
The best analog I can come up with is monoliths in larger organizations are like a manifestation of Amdahl's law. The overhead of communication and synchronization reduces your development throughput. Each additional person does not add one persons worth of throughput when you cross a critical individual count threshold (mythical man month and all that).
I'm not describing this clearly so I should probably actually commit to writing out my thoughts on this in a post describing my experience with this.
We used server side javascript (before Node, I like to think we were ahead of our time, our own fork of Spidermonkey with network and other server code!). We used OPML as a container format with <grazr:script> tags in them analogous to html <script> tags. The "pipes" in our system were much more like unix pipes, where the output of one script could be the input to another. In the running script you could access the feed with a customized "Feed Dom" with items and do manipulations.
With grazr's real-time freshness, feed normalization and database it worked amazingly well. We never were able to get people to understand it (it was super nerdy). I built some pretty amazing apps with it, unfortunately none of it exists anymore :).
Yahoo pipes seemed like the sign of the end for Grazr though. Since we had a widget and feed display tech. a _lot_ of people wired their pipes to grazr's widgets for display and publishing. We saw an initial "burst" on pipes announce and then saw the yahoo pipes usage drop pretty fast and never really recover. As a proxy, it looked like there wasn't enough "there" there to continue to pursue our programming environment.
I still miss it though, it was fun to work on.
This is partially an artifact of history and partially conflating all kinds of 'work' together. We're living in interesting times where there's a shift of what's valuable from simple mechanical and repetivtive skills (industrial revolution type jobs) and knowledge workers and digitial creatives. We have a unique and amazing opportunity, where it is our collective creativity that is what's valuable and not merely our presence or ability to be turn a mechanical crank.
If you follow this advice, and engage in purely a value exchange for the 8 hours a day in something you don't care about and have no real interest in, you're selling 8h/d * 5d/w * 50w/y * 40y = 80k hours. That's a hell of a lot of your life to give over for merely a paycheck, especially when you don't have to. It's that last part that makes me sad, we are in am incredibly unique industry and point in history (which may not last) where you can find a job that engages your whole self while simultaneously making a good living.
I recommend "Drive" by Daniel Pink, about how to think about human motiviation and the opportunities that we currently have to reshape work. Where things like Autonomy (being self directed, choosing your hours and what you work on), Mastery (becoming great at something), and most importantly missing from the rant in this article, Purpose are what really lead to greater happiness and overall better results.
"Purpose" was why I left a high paying job where I was respected to spend 10 years starting companies that (until very recently) meant I made radically under what I was "worth". Would I trade those years? Hell no. Because even if I hadn't had the recent success with Smarterer, I knew I was working towards a bigger and more important purpose, one that could really make a dent in the universe. Being able to do that, and still eat and satisfy all my basic needs is a pretty amazing thing, and one that's totally missed by a rant like this.
When I think about passion, and look for passion in people I want to work with, I look for people who care about what we're doing. Because I don't want "butts in seats" I want people thinking creatively about how we can change the world together. That could mean a day of thinking deeply without banging out any code at all. It could be a crazy "in the shower idea". It could mean long hours one night to hit some big milestone. It also means that if you've been pushing hard on something and you need to take days off to recenter, I trust you because you care about what we're doing together.
Programmer, Developer, Hacker =============================
Smarterer - http://smarterer.com/ A Pluralsight company (http://pluralsight.com/)
Make a difference. Change the world. Do something important.
Be part of a company with an amazing culture: "We are Pluralsight" - https://www.youtube.com/watch?v=wT7eHCCwH84
We are looking for creative web programmers, developers and hackers to join our product team. Ideally you are a versatile technologist who loves to build web products, especially (but not exclusively) if you have Python experience. Our stack is primarily Python, SqlAlchemy, Backbone and CoffeeScript.
Skills are changing too quickly and what makes someone effective in a role or job is evolving. New roles appear daily that no (or few) traditional universities have courses for. People are acquiring more and more skills through non-traditional means. Companies are finding it increasingly difficult to close the "skills gap".
How do you show what you know? How do you even know what you know?
Smarterer has created an innovative, machine learning, crowd-sourced testing platform that will disrupt how people think about skills, testing and learning. Don’t let our name fool you, while our system is fun, our unique, patent-pending technology is also deeply rooted in scientific modern testing theory.
Smarterer was recently acquired by Pluralsight and continues to operate as an independent entity in Boston. We're looking to grow aggressively and have the resources and backing of an amazing organization. Together we're building something truly unique, you should be a part of it!
Interested? Contact me at mikepk@smarterer.com
Smarterer (a Pluralsight company) - http://smarterer.com/ One of Entrepreneur Magazine's Brilliant 100 - http://www.entrepreneur.com/article/233888
Make a difference. Change the world. Have fun. Do something important.
Smarterer gives people unprecedented understanding of what they know and what their teams and organizations know.
We are looking for creative web programmers, developers and hackers to join our product team. Ideally you are a versatile technologist who loves to build web products, especially (but not exclusively) if you have Python experience. Our stack is primarily Python, SqlAlchemy, Backbone and CoffeeScript.
Smarterer is doing something unique and important. We're approaching a point where traditional university credentialing isn't enough. Skills are changing too quickly and what makes someone effective in a role or job is evolving. New roles appear daily that no (or few) traditional universities have courses for. People are acquiring more and more skills through non-traditional means. Companies are finding it increasingly difficult to close the "skills gap".
How do you show what you know?
Smarterer has created an innovative, machine learning, crowd-sourced testing system that will disrupt how people think about skills. Don’t let our name fool you, while our system is fun, our unique, patent-pending, technology is also deeply rooted in scientific modern testing theory.
Smarterer is enabling a revolution: in how the enterprise manages their teams and talent, in the changing job marketplace, and in the way people measure their own skills and share that knowledge.
We were recently acquired by Pluralsight (http://pluralsight.com)! http://blog.smarterer.com/pluralsight-acquisition/ Still driving to the same vision but now with a partner that really gets it!
Interested? Contact me at mikepk@smarterer.com
Edit: I realized after typing that it was confusing... I meant self-similar not similar to duolingo.
Web Programmer, Developer, Hacker
=================================
Smarterer - http://smarterer.com/
Make a difference. Change the world. Do something important.
Smarterer gives people unprecedented understanding of what they know and what their teams and organizations know.
You are a creative web programmer, developer or just all around hacker and want to join a dynamic product team. Ideally you are a versatile technologist who loves to build web products, especially (but not exclusively) if you have Python experience. Our stack is primarily Python, SqlAlchemy, Backbone and CoffeeScript.
Smarterer has created an innovative, machine learning, crowd-sourced testing system that will disrupt how people think about skills. Don’t let our name fool you, while our system is fun, our unique patent-pending technology is also deeply rooted in scientific modern testing theory.
Smarterer is enabling a revolution: in how the enterprise manages their teams and talent, in the changing job marketplace, and in the way individual people measure their own skills and share that knowledge.
We're funded by Google Ventures, True Ventures, Re-think education as well as some amazing angel investors.
Want to know more? Contact me at mikepk@smarterer.com
Web Programmer, Developer, Hacker
=================================
Smarterer - http://smarterer.com/
Make a difference. Change the world. Have fun. Do something important.
Smarterer gives people unprecedented understanding of what they know and what their teams and organizations know.
We are looking for creative web programmers, developers and hackers to join our product team. Ideally you are a versatile technologist who loves to build web products, especially (but not exclusively) if you have Python experience. Our stack is primarily Python, SqlAlchemy, Backbone and CoffeeScript.
Smarterer is doing something unique and important. We're approaching a point where traditional university credentialing isn't enough. Skills are changing too quickly and what makes someone effective in a role or job is evolving. New roles appear daily that no (or few) traditional universities have courses for. People are acquiring more and more skills through non-traditional means. Companies are finding it increasingly difficult to close the "skills gap".
How do you show what you know?
Smarterer has created an innovative, machine learning, crowd-sourced testing system that will disrupt how people think about skills. Don’t let our name fool you, while our system is fun, our unique, patent-pending, technology is also deeply rooted in scientific modern testing theory.
Smarterer is enabling a revolution: in how the enterprise manages their teams and talent, in the changing job marketplace, and in the way people measure their own skills and share that knowledge.
We're funded by Google Ventures, True Ventures, Re-think education, Boston Seed as well as some amazing angel investors.
Interested? Contact me at mikepk@smarterer.com
Web Programmer, Developer, Hacker
Smarterer - http://smarterer.com/
Make a difference. Change the world. Have fun. Do something important.
Smarterer gives people unprecedented understanding of what they know and what their teams and organizations know.
We are looking for creative web programmers, developers and hackers to join our product team. Ideally you are a versatile technologist who loves to build web products, especially (but not exclusively) if you have Python experience. Our stack is primarily Python, SqlAlchemy, and JavaScript with some Backbone and CoffeeScript mixed in.
Smarterer is doing something unique and important. We're approaching a point where traditional university credentialing isn't enough. Skills are changing too quickly and what makes someone effective in a role or job is evolving. New roles appear daily that no (or few) traditional universities have courses for. People are acquiring more and more skills through non-traditional means. Companies are finding it increasingly difficult to close the "skills gap".
How do you show what you know?
Smarterer has created an innovative, machine learning, crowd-sourced testing system that will disrupt how people think about skills. Don’t let our name fool you, while our system is fun, our unique, patent-pending, technology is also deeply rooted in scientific modern testing theory.
Smarterer is enabling a revolution: in how the enterprise manages their teams and talent, in the changing job marketplace, and in the way people measure their own skills and share that knowledge.
We're funded by Google Ventures, True Ventures, Re-think education, Boston Seed as well as some amazing angel investors.
Interested? Contact me at mikepk@smarterer.com
720.790
752.193
717.623
745.300
739.371
694.710
706.695
163.612
724.569
784.664Smarterer http://smarterer.com/
Make a difference, change the world, have fun, do something important.
We're looking for creative hackers and programmers to join our dev team.
Smarterer is doing something unique and important. We're approaching a point where traditional university credentialing isn't enough, skills are changing too quickly and what makes someone effective in a role or job is evolving. New roles appear daily that no (or few) traditional universities have courses for: e.g. growth hacker, social media evangelist, CNC machinist / programmer. People are acquiring more and more skills through non-traditional means. Nascent "education 2.0" initiaitives are exciting, but they still face the problem of measuring and credentialing, especially those that learn online.
How do you show what you know?
Smarterer has created an innovative crowd-sourced testing system and scoring algorithm that allows rapid evolution of content, skill measurment, and question characterization. We've made it fun as well as rigorous, based on modern testing theories.
Putting my money where my mouth is, I've created a software concepts test:
http://smarterer.com/tests/software-concepts
Give it a spin and see how you do. This is the debut of this test so the first few test takers will likely have an advantage. The more people that take the test, the more accurate it becomes. Leave feedback, add questions, share your score!
Smarterer is helping enable a revolution: in the changing job marketplace, in education and learning, and in the way people measure their own skills and share that knowledge.
Our mission: Measure the world's skills
We're funded by Google, True Ventures, as well as some amazing angel investors.
Interested? Contact me at mikepk@smarterer.com
Since the answers do not include $_POST, $_GET is the right answer (although arguably, to make the question clearer, one could specify the method it's not strictly necessary in this question).
(disclosure: I'm the co-founder of Smarterer)
So why did we choose Perl, practical reasons mostly. My co-founder had previously been the CTO of the company that owned Slashdot and he had a team of top-notch Perl people that he thought would want to join us. I had worked a lot with Perl in a previous life so it seemed like a natural fit.
Perl has many strengths. You can "just get stuff done" with Perl. The module library is obviously a huge strength. The first version (effectively a public prototype) of Grazr went live very quickly (days). If you’re careful with it, you can also build some fairly complex projects using Perl. The machinery for Grazr processed a huge amount of data (for the time) multiple terabytes of feed data and had a lot of advanced features like: processing of normalized feeds, an embedded scripting language, cross link analysis, a search engine, and most of the heavy lifting was Perl. Think of the system as an advanced form of the feed API that Google Reader eventually released, nearly real-time (unlike the Reader API we updated feeds upon request) with the addition of a scripting layer on top. I'm very proud of what we built, technology-wise. We failed in other ways (UX, marketing and sales mainly) but the underlying system was quite impressive.
When the feed application language and processing engine stopped being the focus of the company, the Grazr system ran on only a few virtual servers, responsively and smoothly for over a year with steadily growing traffic. That was with the whole company ignoring it and focusing on other 'pivots' (before pivot was the term-du-jour for that). Why the company ignored a service that was steadily growing traffic with absolutely no input from the team/company is a rant for another day. :)
When I co-founded my next company, Smarterer (http://smarterer.com/), I vowed gone-with-the-wind-style that as God as my witness, I would never write a big startup-project in Perl again. Smarterer is written mainly in Python.
Why the Perl hate? First, I still think Perl is a fine language. You can get a lot done with it. But as other's have pointed out, one of the primary design philosophies of the language make it painful in some cases. Large, evolving projects with multiple contributors seems to be the axis of evil for Perl. Startups exacerbate this problem because they exhibit most of these factors and compound it with unrealistic functional time pressure.
What I observed as problems were:
First, with Perl there is always a temptation to be clever. It's a lot of fun, but one of the underlying philosophies of Perl, the "Tim Toady" thing, opens up a lot of temptation to show just how clever you are, usually too clever by half. That always leads to the WTF moments as you’re reading any code you haven’t written in the last week. I started to feel each WTF moment as a tax on our tech team.
When you have a big Perl project with several contributors, coming up with coding standards is a must but usually in a startup environment getting something working is almost always the first priority. Also, coding standards only get you so far with Perl because it's so flexible. If you spend any reasonable amount of time, the unwritten standards of how people write Perl code subtly shifts over time. Usually the code get better as contributors influence each others coding styles, but this makes going back to read working code that’s 6 months to a year old a massive pain in the ass.
Lastly, having coding standards doesn’t protect you against other peoples modules. If you have a large and complex enough system, you’re going to start tripping over the bugs and corner cases in other people’s modules. Perl has the strength of CPAN, but because you can’t enforce a coding standard across all of Perl, debugging into other people’s modules can be an exercise in frustration. When some bug is tracked down to a third-party module, treating it as a black box is no longer an option.
When faced with these problems you can usually work through each case. In the best case it just takes a few minutes of working through the cognitive dissonance, but in other cases you end up breaking out the print statements and/or debugger to instrument code just to figure out what the heck was the intent here.
As a startup guy, the energy required to “re parse” code that’s already been written in subtly different ways starts to drive you crazy because that’s time and energy that could be being applied to solving the bug and tackling the new problem at hand. That re-read and re-parse tax gets worse and worse the large the project becomes and the more people that are involved.
That’s, generally, why I wouldn’t recommend Perl as a language for a startup project, but every case is different. If you're not in the axis-of-evil for Perl, you have incredible discipline and you only rely on a limited number of third-party modules you may be OK.