I'd love to hear about what interesting problems (technically or otherwise) you're working on -- and if you're willing to share more, I'm curious how you ended up working on them.
Thank you :)
Specifically, I spend a lot of time thinking about and writing embedded software. The aircraft is fully autonomous and needs to be able to fly home safely even after suffering major component failures. I split my time between improving core architectural components, implementing new features, designing abstractions, and adding testing and tooling to make our ever-growing team work more efficiently.
I did FIRST robotics in high school where I mainly focused on controller firmware. I studied computer science in college while building the embedded electronics for solar powered race cars, and also worked part time on research projects at Toyota. After graduating with a Master's degree, I stumbled into a job at SpaceX where I worked a lot on the software for cargo Dragon, then built out a platform for microcontroller firmware development. I decided to leave SpaceX while happy, and spent a couple years working on the self driving car prototype (the one that looked like a koala) at Google. Coming up on my third year, I was itching for something less comfortable and decided to join a scrappy small startup with a heart of gold. Now it's in the hundreds of employees and getting crazier and crazier.
Where did you get your masters? I have an EE & CS B.S. from RPI plus 3 years of application development experience in the Fin Tech industry. I am strongly considering swapping industries to Embedded Control--that is what I enjoyed most in college--but I am unsure how to break into the industry. Do you recommend a masters or just sending some apps out? I have a good deal of C++ and micro-controller experience, but none commercial.
There are tons of embedded software projects that lack software engineering rigor. If you're good at unit testing and mocking, for example, there's no reason why you can't unit test embedded code. Applying general software engineering practices to embedded code (effectively) is a good way to differentiate yourself.
Your path to where you are now is crazy...sounds like you've had an exciting career!
The answer to your question is: Yes.
* higher purpose project saving lives and helping people: check
* well-engineered, reliable, smart solutions quadrupling efficiency: check
* planes, Africa, autonomous flight, landing and take off hacks, app based pre-flight checks, oh my!
* all for peaceful, life-supporting, humane purposes!!!
I wouldn't say I am qualified for a job related to embedded programming (even though I know how to code and it's my job) but even if I was there wouldn't be any opportunity for me to bounce between companies like those in a million years.
PS: Sorry for the spelling, not native.
About a year ago, I started spending more time researching about climate change. I learned how important energy storage will be to enable renewable energy to displace fossil fuels. The more I read, the more fascinated I became with the idea of building underground pumped hydro energy storage. I found a research paper from the U.S. DOE written in 1984 showing that the idea was perfectly feasible and affordable, but it seems that nearly everyone has forgotten about it since. (they didn't build it at the time because the demand wasn't there yet. Now energy storage demand is growing exponentially)
A year later, I'm applying for grant funding to get it built. I know that nearly everyone will tell me I can't do it because this or that reason. Because people don't like change and they're scared of big things even if the research shows it makes perfect sense. But I'm doing it anyways because no one else is getting it done. The idea is too compelling and too important to ignore. So here goes nothing!
Again, the most promising option would be to simply attach our installation to an existing reservoir. We don't use any additional water, we just borrow it. For an ample sized reservoir, each cycle would just raise and lower the water level by an inch or so. Another promising option is that we can even use the ocean as an upper reservoir. Salt water can be accommodated -- See our notes about the Okinawa Yanbaru Station.
There are more details in our white paper posted on the website.
Then the total energy capacity is V * rho * g * h, so that energy store is proportional to height, while tunnel boring price is roughly constant as long as tunnel boring volume of the reservoirs is much larger than the volume of vertical shafts.
I realize its a bit oversimplified but if we consider 2 prices: p1 price per volume for boring horizontally (for reservoirs), and p2 price per volume for boring vertically, then increasing the reservoir size by a volume delta V, requires boring 2 * delta V (upper and lower reservoir), while boring vertically the difference in height depends on the diameter...
But it's certainly not out of the question to go deeper instead.
How can I help?
It's really cool to see a feasible alternative to batteries. I think climate change is the single most important problem anyone can be working on right now -- amazing that you've found such a massive lever to pull on this issue.
It's a fair point but I think that you oversimplify the pumped hydro case. Pumped hydro also has quite some electronic components (turbines/motors), electronics, and other moving parts (valves, overflows). I can imagine some of these components require semi-regular maintenance (hence the maintenance shaft in the diagram on the website of OP).
You'd really have to run the numbers to see which costs more to maintain in the long term.
[0]When I say "per unit," I just mean that a huge battery is made up of many cells that will need to be replaced individually, whereas large pumped hydro facilities are still only a small number of total reservoirs.
The pump/turbine technology we use is the same that's been used for a hundred plus years for traditional pumped hydro dams, and the maintenance cost is very low. The life of a project is 40+ years. And in reality it can be 100 years with relatively low amount of maintenance. The San Diego County research posted on our website has good figures on this. thanks!
In reality, when Li-ion batteries are installed in a large system, I believe the round-trip efficiency is quoted much closer to PHS. Sorry, i can't find the best research to cite right now, but here are a couple sources i found with a quick search.
"lithium-based ESS rated for two hours at rated power will have an AC round-trip efficiency of 75 to 85%." https://www.windpowerengineering.com/how-three-battery-types...
https://www.sciencedirect.com/science/article/abs/pii/S03062... "Conversion round-trip efficiency is in the range of 70–80%"
This one says 90-95% https://researchinterfaces.com/lithium-ion-batteries-grid-en...
I've heard that Li-ion installations can get up to 90-95% round trip, which is fantastic, and better than PHS for sure. But it's not the most important detail in the equation. Here's why:
One thing to remember is that power is lost all over the system in conversion and transmission. So raw efficiency can be less important than getting the right capacity to the right place on the grid. And that brings us to cost.
Even though PHS is a little less efficient than Li-ion, 85% for PHS is still really good. (see other my other comment below about 70% vs 85%) And the math shows that investing in PHS is simply cheaper -- even after assuming that Li-ion will drop in price by 3x in the coming decades. This is partly because Li-ion has a much shorter life span and needs to be replaced about every decade.
Li-ion is still great and super important! But it's not looking like the best contender right now for grid-scale storage.
Here are two recent startups in the field with multi-million funding. They were serious approaches. Many people involved with good planning, etc. They still fizzled out when it came to installing their first capacity.
https://www.power-technology.com/news/newsgaelectric-receive...
https://www.greentechmedia.com/articles/read/lightsail-energ...
I believe a reason for the funding problems is the high uncertainty for the economics of storage. Electrical energy is traded in a market. And your trading strategy in the market has a big impact on whether you earn money. Without solid numbers for energy storage and the expected trading outcome, investors will have a hard time.
Rather than a way of making money, I like to think about flexibility and fast dispatching as an enabler for way more renewables to come online, and that is crucial for the human race right now.
I think there's a big gap for both types of storage - fast dispatching for intraday demand variations, replacing gas peakers, and more static storage as in the OP for multi-day gaps in renewable production such as periods of high pressure during winter when wind speeds and irradiance are both very low.
Can't wait to see how this market develops.
How much water do you have to move to power a house, anyway? It must be a lot - a truck pulling a tank of water goes up a hill as part of a journey without worrying about running out of gas, and they could be carrying 40,000 litres or 40 tons of water. Presumably there is no way you could move enough water at home to make a hydro plant - pump it up at night with cheap electricity and run it down at peak time to save money?
> I learned how important energy storage will be to enable renewable energy to displace fossil fuels.
The above is a reasonable statement, however, your website says the following:
> We can’t quit carbon without energy storage. To stop climate change, renewables must replace fossil fuels.
> Without energy storage, renewables will fail to reach even 25% of the energy market by 2040. This will cause global temperatures to rise over 3°C, a level which will cause catastrophic climate damage.
Those are not only misleading but outright lies. Now, I won't hide my bias here: I work on nuclear fission. But here's the reality: there are many possible pathways to net-zero carbon and limiting global temperature rise to well below 3°C (below 1.5°C in fact)
To just list a few:
* Massive adoption of nuclear fission alone
* Development & massive adoption of nuclear fusion alone
* Shift from coal&oil to natural gas, cleaner fossil fuels + scaling carbon capture/sequestration
* Shift from fossil fuels to renewables + storage (probably not alone)
Or any combination of those, in addition to a number of alternative approaches.
---
Edit: Also, it should be noted that the energy sector alone only represents about 1/5 of the emissions problem. In order to get to net-zero GHG and stop anthroprogenic climate change, the clean energy sector needs to expand well past the current global TPES and supply net-zero electricity that allows for the decarbonization of the other main contributors:
Agriculture, steel+cement+plastic, transportation, buildings&appliances, and flora loss leading to lost carbon stores (deforestation etc)
Even if renewables and storage could supply 100% of our electricity or even total power supply, you would still only be 1/5 done solving climate change. There is no unitary solution.
---
Acting as though renewables are necessary, instead of one of multiple options, is denial or malicious. In reality, renewable energy is nowhere near capable of reliably and safely taking on a large portion of our energy supply globally. It is expensive (you can make claims about unit cost, but what really matters is country-scale - look at German electricity prices vs. just about everywhere else), it is dangerous, it takes a lot of land area, and it is the least reliable.
I don't want to spend a lot of time here stomping on renewables, but there is plenty of reason to, and my main point is that I feel it is unjust and immoral for you to claim that renewables "must replace fossil fuels" if we are to stop climate change. It's just not true, and you need to admit that.
The energy industry is arguably one of if not the most important backbone of our modern society, and responsible for the safety and health of billions of people. Whether you're working on the generation or storage side, it is all our responsibility to be honest and make true claims - not to spread biased misinformation when it benefits your particular solution.
I'd like to finish by making it clear I'm very happy you're working on your tech and I hope you succeed in making it the best it can be - renewables are certainly trending to higher adoption and we need reliable, efficient, scalable storage solution in order to avoid dangerous outages and other grid issues.
You bring up valid criticisms of existing solutions, although I do think you should also be fair to those. Most things in life are a trade-off: maybe pumped hydro is a better majority solution for the grid, but lithium ion is an incredibly important, successful and expanding technology that needs to be given credit for its wide range of great applications.
I hope this response has not been inflammatory: I just very much care about maintaining a truthful public discussion around energy. I wish you the best of luck, and I hope you can take something useful from this.
I wonder what the percentage would be like if the energy sector needs to provide enough energy to replace all fossil fuels. It's certainly much higher than 20%.
I'm hoping fission can scale to about 2 EWh annually in the next several decades. Should be noted this is quite aggressive scaling. 500 PWh is more than enough to reach net-zero emissions.
> look at German electricity prices
True, pretty expensive. But they also include capital for investment in energy infrastructure. Such as building lines to get power from the north (high production) to the south (high consumption). The implementation tends to slow, but there are other reasons for that.
Another example is Norway that uses 98% hydro power. Sure, they have topological advantages not available everywhere. But technologies like this could open up more possibilities.
So fission can be utilized, but I doubt that Germany closing plants was a terrible decision.
There are 3 fission fuels occurring in nature: Th232, U235, U238
Actually, our reserves of Uranium are greater (by energy available to generate) than all of our Coal, Oil and Natural Gas reserves combined.
Our Thorium reserves are even greater than those.
In fact, Thorium is extracted as a byproduct of Rare Earth Metal extraction, and so we currently mine enough Th232 per year to replace the entire global energy and fuel industry even though there is no demand for Th232 extraction. Kind of mind blowing.
---
> [fission] it is expensive in deployment
I don't see where this idea comes from - in real life, regions which are powered by more fission have significantly cheaper electricity than those who are powered by less.
---
> the problem with nuclear waste
I genuinely don't think there is a problem with nuclear waste, and that this concern is a myth / misunderstanding based on a mix of fear-mongering via conflation with nuclear weapons and a lack of comparison.
Consider the following: all energy sources have waste products - nothing is 100% efficient.
Fossil fuels pump literally billions of tonnes of toxic gas into the air as their waste product. It moves around, we can't store it, and it is responsible for the deaths of millions of people each year through air pollution.
Renewables production has the same issue (although different gases), and also tends to pollute the water and local environment with other toxic chemicals and metals.
Nuclear fission produces the most dense, least amount of waste of any source, which is solid and easy to manage. We know where quite literally all of it is, and it doesn't hurt anybody or negatively affect the environment in any way as long as you keep it store somewhere.
As far as I'm concerned, nuclear energy does not have a waste problem, it has a waste solution. Global warming is the problem with energy waste, more specifically it is the problem with hydrocarbon waste.
---
> Another example is Norway that uses 98% hydro power. Sure, they have topological advantages not available everywhere. But technologies like this could open up more possibilities.
Agree with you. Renewables tend to vary in effectiveness based on location - in those locations which are well-suited for them, I think they should be used! Though I'm not sure what you mean by "could open up more possibilities" - we've had hydro power for thousands of years.
---
> I doubt that Germany closing plants was a terrible decision.
Note the following excerpt from Mike Shellenberger on Twitter:
Germany’s renewables experiment is over.
By 2025 it will have spent $580B to make
electricity nearly 2x more expensive & 10x
more carbon-intensive than France’s.
The reason renewables can’t power modern
civilization is because they were never
meant to.
A major new study of Germany's nuclear
phase-out finds
- it imposes "an annual cost of
roughly $12B/year"
- "over 70% of the cost is from the 1,100
excess deaths/year from air pollution
from coal plants operating in place of
the shutdown nuclear plants"And Germany has much to do for carbon efficiency, but for total emissions it is somewhere in the middle.
https://file.scirp.org/pdf/ME20120500016_67195744.pdf
Data is for overall efficiency, not power production.
And Shellenberger is a nuclear lobbyist for that matter and his statements should be scrutinized. I am not fully content with the decision to make such a cut for fission power generation, but all these numbers are conjecture.
I think it is extremely foolish to make caricatures of people. Twenty years ago, Elon Musk was a software startup guy who had no idea about anything hardware - but that's only because nobody bothered to consider the full human behind the caricature.
Mike Shellenberger was an anti-nuclear activist for much of his early life and has always been (and is still) an environmentalist. Furthermore, he may be a lobbyist now (I'm not sure if you are right or wrong), but he ran for governor of California a few years ago. He has been very explicit in explaining his reasoning for shifting from anti-nuclear to pro-nuclear in multiple talks and articles.
Take a look at the full human, and your justification for scrutiny fades away. Everybody should be scrutinized to an extent, but he is not fundamentally a biased lobbyist with financial incentives.
> Germany has much to do for carbon efficiency, but for total emissions it is somewhere in the middle.
This is the problem, man. Germany has spent hundreds of billions of dollars on renewables and they still have high GHG emissions - all they have to show for their massive spending is a couple thousand extra deaths per year and higher electricity prices.
If you gave my company the same amount of money, we'd have the entire world to net zero emissions within two decades.
Goes to show the inefficiency of government funded programs, and the awful incompatibility of renewable energy with a reliable, affordable consumer electricity market.
> I like to use current numbers, because extrapolating development is often pretty close to lying.
We can use current and past numbers: for its entire existence, nuclear fission has been the (a) safest, (b) highest fuel density, (c) least waste-producing, (d) lowest emissions, (e) most reliable mass energy source humanity has ever had.
The new generation of reactors will only improve this divide between fission and everything else. If you are against extrapolating development and want to rely on established numbers, you must conclude [fission > renewables]
I know I'm biased, but I'm also right about all those superlatives.
If you want to send me research supporting some of your thoughts here then I'd love to see it. I do know for example that it's a very valid debate whether or not nuclear has a place in our climate fight.
I'll try to re-work the language in my materials to make sure I'm not excluding other valid viewpoints. Thanks!
but none of the below are renewables
* Massive adoption of nuclear fission alone * Development & massive adoption of nuclear fusion alone * Shift from coal&oil to natural gas, cleaner fossil fuels + scaling carbon capture/sequestration
Parent is primarily disputing: "To stop climate change, renewables must replace fossil fuels." and if renewables fail, "this will cause global temperatures to rise over 3°C"
affordable, but efficiency is so-so. 70-80% according to Wikipedia [1]
[1] https://en.wikipedia.org/wiki/Pumped-storage_hydroelectricit...
Say what? So-so? 70-80% efficiency sounds pretty damn amazing!
And actually, we think that 80-85% round trip is more accurate for our projects because we'll use the latest/greatest tech (variable-speed reversible francis style pump/generator turbines). I think the 70% from these figures is citing older projects with pump/turbines that were not quite as efficient.
I wish you luck with your endeavor. I think you are correct that the problem of energy storage is the most important one to solve to allow renewables to really take off for general power supply.
I also believe that the efficiency of storage is secondary to a degree as renewables can supply an enormous amount of energy. So loss from pumps or energy conversions are nearly insignificant. At least at the moment where energy storage is in such a bad shape. Certainly a lot more promising than having batteries everywhere.
I don't have his spreadsheet anymore, but it seemed really solid.
If you can remove the mountain you could scale this out to every one in the world and single handedly solve this problem.
There will be other problems to overcome but someone will figure it out why not you? I wish you all the best in this very important effort.
The challenge is obviously scaling, since every municipality is different. For now it's going to cover my region and we'll see from there.
In my one county alone there are 90+ municipalities, each with it's own Planning and Zoning Commission, and most with their own (varying level of) website. I'd say 5-10% don't have a website either.
In your situation, how are you getting data for when land is up for sale/zoning etc.?
I tried something similar but mostly to figure where the land is being purchased recently in a region. But then land/parcels/addresses system is all over the place and, even that info is not consistent across cities.
have you looked at data providers who may have this data?
Both sound like interesting problems, and it would be awesome if municipal-level land data was available at scale.
"Forests within 100 miles"
"High rises within 10 miles"
"Anything within 0.5 miles"
DNS is currently centralized and controlled by a few organizations at the top of the hierarchy (namely ICANN) and easily censored by governments. Trust in HTTPS is delegated by CAs, but security follows a one of many model, where only one CA out of thousands needs to be compromised in order for your traffic to be compromised.
We're building on top of a new protocol (https://handshake.org, launching in 7 days!!) to create an alternate root zone that's distributed. Developers can register their own TLDs and truly own them by controlling their private keys. In addition, they can pin TLSA certs to their TLD so that CAs aren't needed anymore.
I wrote a more in-depth blog post here: https://www.namebase.io/blog/meet-handshake-decentralizing-d...
¹ http://www.lifewithalacrity.com/2016/04/the-path-to-self-sov... ² https://w3c-ccg.github.io/did-primer/
There is an issue though — the auction system gives early advantage in buying names for cheap. If only 100 people are buying names on day 1, they’ll be able to buy a lot of the names without competition. Handshake has a mechanism to prevent this. Names are released for bidding over the first year so that people who learn about it six months late can still register good names. The release schedule is basically a hash(name) % 52 to determine which week you can start registering any name.
While in college (CS & Math), I got heavily interested in growing food in the most efficient and healthiest way possible. I was a dreamer when I started so I thought more of how to grow 'earthly' produce on Mars, but then I realized that my own planet Earth is so massively underserved.
It's basically like this- I mastered growing leafy greens in indoor closed environmenet, then I tried to cover all the major physical and biological markers, then I try to optimize the most optimal levels of 5-6 variables (currently) that I can fully control and may produce the best phenotype- CO2, O2, Light, Nitrate, P, K. These parameters have their own sub definitions.
So far I have had great results. I am trying to raise investment so I can finally make it a reality. Check the numbers here: hexafarms.com (no fluff)
How's the taste?
Not denying it's possible to grow food very efficiently indoors but a vastly oversimplified opinion is that plants need sunlight to be tasty. Is this wrong?
Yes, you don't really need sunlight whatsoever. I was myself shocked until I recalled high school biology concept of genotype and phenotype i.e. the genetic structure that manifests itself given the right physical conditions (at least of plants.) As for the plants' nutrients, here's a classic- Teaming with Nutrients: The Organic Gardener's Guide to Optimizing Plant Nutrition, by Lowenfels. I was amazed to find how complex, yet simple plants are.
I love this! Makes me happy to see someone's working on such an interesting problem that would benefit many.
For feedback, I believe using photographs of the leafy greens would be effective in communicating your vision.
I actually graduated from college this year; and for personal reasons I've had to change countries; now I'm in another Master's program... ready to drop out anytime. The whole project has been dead for months once in a while! I'm more trying to leverage ML for optimizing things. I guess that's what modern farming is missing (not ML per se, but optimization).
I'm trying to raise some investment (or in the worst case bootstrap and risk everything in the next few months), then I will go crazy with the idea.
and Beanstalk (a YC company) https://www.beanstalk.farm/
Thanks for the references.
Hab 1 had aquaponics and fish, not sure where Hab 2 is going to look like as they haven't shared much but he's just started churning videos out again the past month or two.
It's a really neat project, I just hope he continues to show as much as he did with Hab 1 now that he's part of a startup.
Are your farming systems fully automated? If so, has that been more of a software challenge, or more of a mechatronics challenge?
My vision is to have distributed farms (as opposed to conventional wisdom, i have found that smaller indoor farms will be more profitable) every eight blocks or so.
Not really- It's quite manual (as of now). I had to change my country almost three times since I started; so I'm rather focussing more on data, and training algorithms part to figure out the right parameters (and the farm is a just a testbed). One example would be to have a $5 camera for measuring growth than buying a $100 3D what not camera.
Is it possible to setup a 'microfarm', similar to a window fridge appliance, in a part of an apartment room?
I'm ok with some manual work every 2 days, such as filling in a water container.
Besides water & substances, how much electricity would this use to grow a generation of leafy greens, per kg of produce?
Thanks for working on this!
johndoe.shopify@daily.paced.email
johndoe.stripe@weekly.paced.email
johndoe.github@monthly.paced.email
I'd love to hear how you'd use it.
> At the end of each period, a single email is sent to the real email address containing all of the messages the alias received over that timeframe.
Why not send each received mail individually? If you aggregate them first, it makes it very difficult to reply to individual messages with standard email clients.
I use a similar but far less fancy approach with email filters: I have everything put into its own filtered folders then only check them on a schedule.
Your approach is good because the schedule is right there in the email address.
Food for thought.
I mean use github@johndoe.paced.email And have an admin ui that lets you set "github@johndoe.paced.email" =>"weekly"
Then you don't even need a website.
Across all platforms (not just Reddit), people including myself like to save/bookmark interesting content in the hopes of getting some use out of it later. The problem arises when you start accumulating too much content and forget to ever check that stuff out.
I'm working on a solution to help resurface Redditors' saved things using personalized newsletters. I'm calling it Unearth and users get to choose how frequently they want to receive their newsletter (daily, weekly, or monthly). The emails contain five of their saved comments or things and link directly to Reddit so that when viewing it, they can then decide whether or not to unsave it.
Basic functionality is all there, just needs some more styling and the landing page could be spruced up.
Feel free to try it out and let me know what you think or if you have any suggestions.
Kinda different, kind of the same but i'd love to use an app with much better search than the 'direct search' currently in most aggrogrator/ note apps. If i searched 'quotes' it would rip out and return all the things in italized, in quotes, or things that the algorithm deems as quotes based on it's scrape of the internet; Kinda like google but 'personal search' based on my notes, articles, all my different emails (work, and my 37 different gmail accounts) and websites I frequent (like reddit, hacker news comments, etc.) There was an HN article the other day that got me thinking about this problem, but i can't seem to find it. However, it approached it from a much deeper technical level, utilizing emacs and searching through his code. If you could bring that into an easy to use, consumer facing GUI I think it'd have potential to be pretty game changing.
'Personalized Search, and we don't have to steal your data because you willingly give it to us' - Google
And that's a really interesting idea regarding search. Would love to see the HN thread/article you mentioned to get a better understanding of the concept. As of now, Unearth's only focus was on active content resurfacing, but I've seen many Redditors mention the wish to search their saves as well so I think I'll look more into this.
Appreciate the ideas, keep them coming.
For privacy you needn't require the reddit ID of your users. Simply that they want to save something from reddit to their tryunearth.com account.
> Simply that they want to save something from reddit to their tryunearth.com account
When you say that, I envision the extension overriding or extending Reddit's save button functionality by making an API call to the unearth backend. Is that kinda what you had in mind?
I would need to figure out how injection would work for saved comments, do you have any ideas? I'm definitely going to save this idea so thank you!
I take a ton of notes on Notion, but I worry that I'll never see most of them again. Maybe part of the value is just in writing the notes in the first place...?
Kudos for solving out this problem for Reddit!
Why not call it Digg?
It's funny how we're all working from different definitions of the word "problem" - I'm certainly not changing the world with medical supplies for developing countries, renewable energy, payment systems and so on.
But it's something I'm really passionate about, and I'd be over the moon if I came anywhere close to the picture I have in my mind.
Back when I was studying German and Chinese, I would spend hours and hours on rote practice with little to show for it. My brain almost felt like it was on autopilot - the eyes would read the words and the hands would write the sentences, but the neurons weren't really firing. It didn't feel like I was properly building the synaptic bridges necessary to actually use those words in conversation.
On the flipside, after just 20 minutes speaking with a tutor, my proficiency would improve leaps and bounds. Being forced to map actual, real-world thoughts/concepts to the words/expressions I had learned - that's what made everything clicked. It felt like the difference between just reading a chapter in a maths textbook, and actually doing the exercises.
So after keeping track of progress in NLP and speech recognition/synthesis in recent years, it seemed like a logical time to start. Progress is slow/incremental, but it is there.
2.) Porting my Python code for nonlinear gradient driven optimization of parametric surfaces to C++. Includes a constraint (propagation) solver based on Minikanren extended with interval arithmetic for continuous data (interval branch and contract). This piece is a pre-processor, narrowing the design space to only feasible hyper-boxes before feeding design parameter sets (points in design space) to the (real valued) solver. Also it does automatic differentiation of control points (i.e. B-spline control points) so I can write an energy functional for a smooth surface, with Lagrange multipliers for constraints (B-spline properties). Then I get the gradient and Hessian without extra programming. This makes for plug and play shape control. I am looking to extend this to subdivision surfaces and/or to work it towards mesh deformation with discrete differential geometry so I've been baking with those things in separate mini-projects.
3.) Starting the Coursera discrete optimization course. This should help with, e.g. knapsack problems on Leetcode, some structural optimization things at work, and also it seems the job market for optimization is focused on discrete/integer/combinatorial stuff presently so this may help in ways I do not foresee.
4.) C++ expression template to CUDA for physical simulation: I am periodically whittling away at this.
I like other methods of getting local control, or finer shape control of surfaces. In my stuff I've used truncated hierarchical B-splines (THB-splines), which are great for adding detail, but useless for changing topology. People speak highly of (analysis suitable) t-splines but I say they are complicated and subdivision may be better overall now anyway. Generally speaking, I think the whole industry will have to go to subdivision. (Among friends I'd say it may carry right down to poly meshes via differential geometry but those two representations might play well together given the right tools)
Reading recommendations:
For everything you ever wanted to know about a B-spline, including a C++ library implementation from scratch, highly documented and explained: 1.) Piegl and Tiller "The NURBS Book" This includes a tiny bit of shape control via optimization.
For an explanation of the basics of B-spline nonlinear optimization with Lagrange multipliers, focuses on ships, there is a chapter here that takes you to the state of the art, circa 1995: 2.) Nowacki, et al., Computational Geometry for Ships
3.) Tony De-Rose's book "Wavelets for Computer Graphics" actually has some good scripts getting at the basics of wavelet B-splines and some facets of hierarchical parametric geometry.
The above is a start at form parameter design for B-splines. This was okay 20 years ago. It's still importatnt as a basis for understanding optimization of parameterized shape. ---Even subdivision surfaces have control points.
Generally B-splines were found not to be flexible enough for representing local details efficiently. Further, the optimization techniques still require a lot of manual setup to get things right...
The next steps are still in development: -subdivision surfaces are a way forward for shape representation. Generally they were more problematic for computing engineering quantities of interest, especially and precisely where they "go beyond" the B-spline to allow surfaces of greater flexibility -- that is where the analysis suitability breaks down to some extent. Again, this has been patched up in the last couple of decades but still change is slow to come to the engineering industry.
I think it's well worthwhile to look at geometric optimization in computer graphics as well. See The cal-tech multi-res group, Keenan Crane at CMU (geometry collective), and tons of siggraph papers where discrete differential geometry has been leveraged to do neat things with shape. (E.g. curvature flow: https://www.cs.cmu.edu/~kmcrane/Projects/ConformalWillmoreFl... I think there is newer work building off this and adding more complicated constraints but I can't remember off hand. As is they have some already!)
Back to the point: you wanted optimization readings. Well it's mostly in the literature, and the literature is mostly kind of vague when it comes to parametric optimization of B-spline. Though the high points are mentioned, the detail is often hardly much better than you find in Nowacki, 1995. To this end, I have some really specific entry level PDFs that might help, and the first part of my stuff is written up in this paper: https://www.sciencedirect.com/science/article/abs/pii/S01678... This deals mostly with curves, but has a direct extension to surfaces. Automatic differentiation really helps here! (I never published this bit on the extension to do surfaces directly (with all their attendant properties as potential constraints) as my professor said "direct surface optimization was to expensive". Looking at the discrete differential papers as of late, I tend to disagree. )
This would be pretty hefty (Expensive) for a mesh. I’ve used it successfully for splines where a smaller set of control points controls a surface. Mesh direct sounds expensive to me. I assume you looked at constrained mesh smoothers? (E.g. old stuff like transfinite interpolation, Laplacian smoothing, etc?). Maybe newer stuff in discrete differential geometry can extend some of those old capabilities? What is the state of the art? I have a general impression the field “went another way” but not sure what that way is.
As for the auto diff, I’ve also got a version that does reverse mode via expression trees, but the fwd mode has been fast enough so far and is very simple. Nice thing here is that overloading can be used to construct the expression tree.
Of course if you do only gradient optimization you may not need the hessian. It’s there for Newton’s method.
I use FEMAP at my day job have found Laplacian smoothing and FEMAPs other built in tools have been wanting.
I am currently thinking that my goal is to try and use reinforcement learning to build high quality meshes. In order to do that you need a loss function and if you are building a loss function you might as well wrap an optimizer around it.
FEMAP Seems a hot topic these days. Some folks at my work are building an interface to it for meshing purposes.
Most of the research is being done out at the Colorado school of mines by Paul Constantine. The basic idea is that you reduce your parameter space to the eigenvectors of the sensitivity matrix with the largest eigenvalues. Some of the work I have seen in constitutive modeling (and UQ) has effectively reduced parameters spaces of a couple hundred DOF to about 5-6.
Yeah Colorado School of Mines! Small world, I am in the metro area. I've actually talked with a physics proff from there about helping with a project.
Seems like that would be the (or a function of the) thing we are after in sensitivity analysis.
On the other hand, it appears that I may be able to get away with some naive assumption about this quantity, compute eigenvectors and find the active subspace... and then vary the mode in these directions.
Is this for local or global optimization?
Part of my stuff was about finding a way to guarantee that a particular set of inputs results in a feasible design. (Edit: maybe active subspace could replace this... or exclude poor regions faster)
The other part (the gradient driven part) solves curves and surfaces for shape which conforms to constraints. We really need the fine details to match as the constraints are often of equality type.
From there, it seems this active subspace method could really help in searching the design space. (From what I read, this is the purpose) A more efficient method of response surface design. My stuff is agnostic about this.
Then again, surely it could be of used in more efficiently optimizing constrained curves and surfaces... I will keep thinking but it seems a secondary use at best, or would you agree?
Active subspace comes from the uncertainty quantification community. If you assume all your parameters are Gaussian, then the sensitivity matrix is directly correlated to the probability density functions. I find it easier to think in terms of the sensitivity matrix, but useful to realize the sensitivity matrix to approximate (complex) probability distributions.
My though was that if you were optimizing have a huge parameter space theta = [theta1, ... thetam] then you could reduce the parameter space by only looking at theta_reduce = [theta | d loss/d theta > threshold] or you could look at active subspaces and change the parameter space to xi = [xi1, ... xi_m] where x_i = SUM a_j theta_j.
xi_i could be given by the largest eigenvectors of the sensitivity matrix S_ij = d^2 loss/dtheta_i dtheta_j
Wouldn't it be nice if hacker news supported latex.
I haven't done any work here, but I suspect I will be doing some of this towards the end of summer.
Simply for the experience. C++ is more in demand right now, as far as I can tell, sorry to say.
Build a functional to describe your ship problem, minimize it: if the solver is happy, you have a boat.... uh, or if you haven’t solved the entire problem, you have some geometry which can be stitched together with more optimization to make a boat.
More broadly, “why a boat?” Answer: because boats have a lot of constraints, and a lot of shape ( Gaussian curvature, non rectangular topology, a need to be cheaply produced, etc etc)
So it’s a good problem to tax your generative design or design space search/optimization capability.
We've solved scaling and reliability (we handle 20 billion API requests a month), and we're now focusing almost all our efforts on our data quality, and new data products (like VPN detection).
We're bootstrapped, profitable, and we've got some big customers (like apple, and t-mobile), and despite being around for almost 7 years we've still barely scratched the surface on the opportunity ahead of us.
If you think you could help we're hiring - shoot me a mail - ben@ipinfo.io
- We're super developer friendly - you don't even need an access token to make up to 1,000 requests per day. We have a clean / simple JSON response, and official SDKs for most popular libraries
- We have a quick, reliable API. We obsess over latency and availability, and handle over 20 billion API requests a month. (here's a technical overview of how we reduced rDNS lookups by 50x: https://blog.ipinfo.io/reducing-ipinfo-io-api-latency-50x-by...)
- We obsess over data quality. We have a data team that's constantly striving to make our data and accuracy even better than it already is.
- We're innovating. We've launched and are working on exciting new data sets and products in the IP and domain data space (VPN detection, the host.io domain API, and more).
- We care about our customers. We have people working on customer support and customer success. If you run into an issue or need help, we'll be there to answer your questions.
On the side, I'm an advisor to an impact investment foundation that is expanding their operations to East Africa. They're setting up an investment fund and accelerator programs to help companies tackle development challenges.
I'm also involved in a startup that is working to develop a new fintech app to create more data and access to credit for small-scale businesses in East Africa. It's a basic PWA app, not released yet, which has some real potential of scaling up and addressing some pretty substantial development challenges. (If anyone is really good with writing a bare-bones PWA based on Gatsby optimised for speed and low-bandwidth environments, please give me a shout).
I've had a weird career. Started out as a programmer in the late 90's, did my own startup in the mid 00's which was a half-baked success, moved to Africa for a few years and worked for the UN, moved back home and had kids, moved back to Africa and worked as a diplomat covering lots of conflicts in the Great Lakes region, moved back home again, worked for the impact foundation for a year and then rejoined diplomacy to do cyber work.
I didn't know any such norms existed. What are some of the existing agreements, and if you can talk about it, what are some of the new ones you're trying to push forward?
Your career sounds crazy...in a good way! Was your initial involvement with the UN in a technical role?
Basically, it's about trying to defend international norms from an onslaught of attempts to make states the primary defender of the informational realm, and thereby legitimise opression.
Yeah, first job for UN was coding a shitty CRUD system in order to keep track of HIV infections in East Africa.
My rationale for starting this project was that I like specific features or facilities of many individual languages, but I dislike those languages for a host of other reasons. Furthermore, I dislike those languages enough that I don't want to use them to build the projects I want to build.
I'm still at a relatively early point in the project, but it has been challenging so far. I'm implementing the compiler in Crystal, and I needed a PEG parser combinator library or a parser generator that targeted Crystal, but there wasn't a PEG parser in Crystal that supported left recursive grammar rules in a satisfactory way, so that was sub-project number 1. It took two years, I'm ashamed to say, but now I have a functioning PEG parser (with seemingly good support for left recursive grammar rules) in Crystal that I can use to implement the grammar for my language.
There is still a ton more to be done - see http://www.semanticdesigns.com/Products/DMS/LifeAfterParsing... for a list of remaining ToDos - but I'm optimistic I can make it work.
Did you use Sérgio Medeiros' algorithm for left recursion, perchance?
My takeaway from Tratt's explanation was that the general technique of growing the seed bottom-up style when in left-recursion - I think I've also seen that idea termed "recursive ascent" somewhere else but I can't place it offhand - seemed reasonable, so that's what I kept working on until I figured out something that seemed to work.
Later on, I ran across https://github.com/PhilippeSigaud/Pegged/wiki/Left-Recursion, which describes Sergio Medeiros' technique at a high level. One of the nice things I used from the Pegged project was the unit test suite. I re-implemented some of the unit tests from Pegged in my own PEG implementation and discovered that it failed at those unit tests.
It took me another number of months to figure out why my implementation failed the unit tests. I re-jiggered my implementation to make it handle the scenarios captured by those unit tests, and then naively thought "hey, it works!"...
All was well until I ran across another set of unit tests in the Autumn PEG parser (see https://github.com/norswap/autumn). My implementation failed some of those as well. After another number of months, I had a fix for those too.
Long long long story short, this process continued until I couldn't find any more unit tests that my implementation would fail, so once again I'm at the point where I think "well, I think it works".
There have been a number of occasions where I've thought "if this doesn't work, I'm just going to re-implement Pegged in Crystal!". Perhaps that's what I should've done. Ha ha! In a few months, when I find another test case that breaks my implementation, I may just do that. We'll see. I hope it doesn't come to that. Fingers crossed. :-)
I think V is an impressive language, but it isn't quite geared toward my vision of what a language ought to be.
I am more a Rubyist than a C, Rust, or Go developer, and so my preference is for a higher level language that's a little more pleasant to use and doesn't make me think about some details that I consider "irrelevant". I'm firmly in the "sufficiently smart compiler" camp, and think that I shouldn't have to think about those low level details that only matter for the sake of performance - the compiler ought to handle that for me.
- Relevant to you and your interests...
- ... but diverse enough to feed your intellectual curiosity
- Delivered in a timely fashion: apart from once a year big events, most things can wait for a few days, no need to require you to read the news every day
- Include some analysis to allow you to see the big picture
When I started a few years ago, I thought naively that a little machine learning should do the trick. But the problem is actually quite complex. In any case, the sector is ripe for disruption.
Your list of points is great, if you can figure out a way to deliver a service like that it would be incredible.
I think one of the biggest challenges for current publications is the tie to advertising model - advertising business model forces products to decrease in quality over time. Same thing happening to Google and Facebook, but super apparent in news sites. They're fucking awful these days, I can't read a single article without ten huge popups and a paywall.
All the points mentioned in the parent comment has been done before: magazines and newspapers. (Some) people used to subscribe to multiple publications to get their intake of information. The wide ranging, impact based news is the daily publication specialty. The newest in your specific interest is the magazines' playing field. News special reports used to be in longform and discusses all the finer points, including analysis and graphs to see the big picture. Magazines with themes serves the intellectual curiosity.
Somehow in the age of niche creators, these companies die out. I think the saying 'the sector is ripe for disruption' is true, but not in the way of software or automation. Better business model is really needed. The business model has been done before; the evolution to bite-sized factoids is the consequences changing to more heavily advertising-based business model.
The limiting factor of paper space and physical distribution seems to strike a balance: news to be printed and distributed need to be worth it for the public to pay. Maybe bundling also made it work[0]. The specific 'small' niches in newspaper/magazines can be fulfilled by sharing the cost with the mass of subscribers.
There is a tradeoff in the wide influence of gatekeepers, but even in that time independent publications managed to survive.
I think finding this balance again is really the key. Should we go back to tax-funded publications? Or will people welcome a microtransaction for articles? Or should the publications deliver curated, less frequent summaries to make customers happy? I think the disconnect between the customers and paying for content is driving down the quality and demand (in revenue) of these publications.
The recent years have shown that subscribing to the publications themselves are not optimal. Putting up a paywall angers people, but The Guardian have never wiped their donation banner off their pages. The need to find the correct business model for publication is urgent for the masses too; democracy that actually follows the people's will depends on this.
I don't follow the current landscape, but what The Athletic is currently doing is pretty interesting for sports.
Me, personally, really like the 'The Espresso' concept from The Economist. They curate 5 stories each day and deliver it in the morning directly in the app. No space to switch tabs and disengage but space to dig deeper in the story through the links.
[0]: https://cdixon.org/2012/07/08/how-bundling-benefits-sellers-...
The website is all in Dutch, but you can probably get the gist of it (I live in the NL but don't speak Dutch, but their mission is quite clear).
We're making sure journalists get the best tooling to make their work. By empowering them, we help them spend time on what's actually important: writing quality content.
Would love to exchange some ideas with you.
The goal is to have a system that avoids the rich-get richer effect, avoids false negatives (good content with bad rating) and in general a better correlation between votes,quality and clicks than upvoting systems.
I wrote a small simulation to test my hypotheses against HN and reddit scoring mechanisms, and it looks very promising.
Unfortunately I don't have more time to work on it...
https://3dprintingindustry.com/news/3d-printing-impact-on-de...
EDIT: Sorry, I missed the reviews part. Do you mean easily getting a second opinion based on diagnostic imaging?
Edit: Not in US, but building planning to launch there. You can't practice dentistry in US if you haven't got US diploma. However, diagnostic dental work (at least in some states) is exception to this.
What work, generally?
For some reason I keep hearing about people flying to Serbia to do this.
An ex-dentist attempted to strong arm me into a receiving an occlusal adjustment because my TMJ popped during a single visit. I knew this permanent procedure is rarely the best solution for the scenario. The dentist subsequently became irate and told me, "You'll lose all your teeth and look like an AIDs patient!" You can probably guess what era he's from.
I wanted to file a complaint, but it would've been my word against his, his assistant, and his hygienist. Absolutely ridiculous situation. It also provided a snapshot into how medical professionals exploit patient ignorance for revenue.
This eliminates so much fraud and mistakes.
However, it's a bell curve. There are extremely moral and extremely immoral people. Some of them are dentists.
Absolutely true. However, it seems that other areas of medecine have better systems in place to prevent abuse, and dentristry would do well to follow suit.
Let's focus on the second part of that statement. It means that majority of cost of dental care goes to the practitioner, rather than to drug makers. This means they have more reason to cheat. The payoff is higher.
Research that showed the 28% figure: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3036573
I’m actually a dental student myself, and it saddens me that a significant chunk of dentists take advantage of the of the self-policing inherent to the field. It generates generalized distrust and resentment among the rest of dentists, in addition to being simply unfair.
As far as I know, there are no diagnosis codes in dentistry, just treatment codes. If it were, I imagine it could be possible to prevent this problem by randomly and routinely validating patients charts.
On a side note, it is a budding dream of mine to build a start up related to dentistry, particularly in the realm of dental informatics, but not limited to it. I was wondering if you would be willing to chat with me about your experience sometime. It sounds fascinating.
Thanks again.
Let's say you are Delta Dental, these 28% are basically an insurance fraud. If you could get rid of it, you would save billions. You could offer lower premiums and full coverage without any copays.
Started because of frequent multitasking heavy work with limited resources.
Open Beta (macOS) as soon as I finish license verification and delta updates.
The basic idea, on and off for close to five years. Started out experimenting with shell session persistence (solved[0], but not quite), then prototyping a browser-concept and playing with browser-extensions, then settling on the OS-level...
Not really a smooth experience in my opinion, doesn't map quite as well to the concept of "working context" as I think of it. Also, you'd have to maintain your list of users, and manually sync any settings, etc. - whereas with Cleave, I'm planning on implementing white- or blacklisting of applications on a context-basis (and system settings etc. are implicitly shared).
> Separate user accounts is kind of the naive (and not quite complete) solution to the same problem.
I too have attempted to solve this problem with user accounts; and yeah it doesn't work well. Files are a pain to share, the log-out-log-back-in process takes forever, and a bunch of preferences don't sync across user accounts.
I particularly like the idea of having a super-low-energy mode where it's just for writing or reading, and saved states for my countless research sessions. Also, being able to freeze my dev workspace and resume it any point sounds amazing.
Excited to try it out!
I'm on Linux, so I won't be able to use your app, but great idea and good luck!
Also it'd would be a data integrity nightmare because if one context shared the apps from another. How would you manage memory corruption, and allocation and saving in this sort of scenario.
Anyway, sounds awesome.
Good luck
- Getting a representation of a city that cleanly divides paved areas into distinct roads and intersections, and understands the weird multi-part intersections that Seattle has plenty of. [This](https://docs.google.com/presentation/d/1cF7qFtjAzkXL_r62CjxB...) and [this](https://github.com/dabreegster/abstreet/blob/master/docs/art...) have some details about how I'm attempting this so far.
- Inferring reasonable data when it doesn't exist. How are traffic signals currently timed? No public dataset exists, so I have heuristics that generate a plan. If I can make the [UI](https://raw.githubusercontent.com/dabreegster/abstreet/maste...) for editing signals fluid enough, would it be worth trying to collect this data through crowd-sourcing?
- Figuring out what to even measure to demonstrate some change "helps a bus run faster." Should people try to minimize the 99%ile time for all buses of a certain route to make one complete loop over the whole day? Or to reduce the worst-case trip time of any agent using that bus by some amount? Or to minimize the average delay only during peak times between any pair of adjacent stops?
- Less technical: How to evangelize this project, get the city of Seattle and advocacy groups here using it, and find contributors?
you can then think about optimizing routing for commuters, or late night service around a random event with increased traffic, etc.
Map data comes from OpenStreetMap. Aside from the problems I mentioned with inferring good geometry for some intersections, it's pretty solid.
Drivers in A/B Street have to find parking, and walk between buildings to parked cars. So the amount of on- and off-street parking in an area is pretty important; there might be lots of congestion in the sim if I don't know about parking. I'm inferring on-street from http://data-seattlecitygis.opendata.arcgis.com/datasets/bloc..., which is extremely inaccurate, and I have no data source yet for private parking attached to individual buildings.
Until I have some of this data, calibrating the sim against reality is pretty tough.
All because I can't get a 3DS game to load modified videos.
Some excellent free reverse engineering courses for Windows[0] and Linux[1] were posted in another HN thread recently, and a couple other HNers and I started a study group for the Linux course. We have a Discord group, and we're meeting for the first time on Thursday, in case you're interested. (It sounds from what you're doing as if you might already be past those resources, knowledge-wise.)
[0] https://github.com/0xZ0F/Z0FCourse_ReverseEngineering [1] https://beginners.re/
Having played around a bit with a jailbroken Switch, the primary reason is to be able to either run mods for games (e.g. there's a surprisingly viable set of tools for converting/bundling Skyrim mods for PC for use on the Switch version) or to run homebrew apps, the common ones being emulators. Previously, I had played around with GBA emulators on PSP/PS Vita, but the Switch form factor and screen size are a lot nicer to play on.
That being said, you could also go the route of putting a completely foreign OS instead of just modified Switch firmware. Android seems to run flawlessly on a Switch from my testing; even though Android doesn't seem to be able to access the joy cons directly when they're attached, you can still pair them with bluetooth and they work fine even when they're slotted in. Android even seems to have pretty good built-in support for that kind of input device; you can easily navigate through the icons on the home screen, hit "A" to select whatever's highlighted, etc.
You were trying to load VR porn on there, huh?
- initially, all manual
- secondly, timers - I know when some airlines do deals, so I go look
- thirdly, I found other sites indexing unusually cheap flights, but they're not always the same price on my site
- fourth, built a script to search my own site for a route, but the number of combinations rockets with the increase in date ranges. If you're taking different stopovers etc, it becomes ludicrous.
- it's growing at least, but finding ways to make it less hands on and less mind-numbing is a never ending quest. Although I still enjoy it :)
[0] http://www.ai.mit.edu/courses/6.034f/psets/ps1/airtravel.pdf
I've found, however that for one way routes I can do fairly well, and for return journeys I have to just sample the space and hope that I'm hitting some good dates in there, currently.
There are rate limits on a lot of tools/APIs as well, so if I can narrow my search boundaries quickly with some hand tweaking, it helps a lot.
I always wondered how/if finding cheap flight deals could be automated, and it's interesting to hear that it's as hard as it seems.
Just FYI, your logo and part of the menu disappear off the sides of the site background on my 4K screen.
I think it's just that with the background image being 1880×1253, it isn't wide enough with background-size: auto. I just changed it in FF devtools to background-size: contain, and it looks fine.
[0] https://www.dropbox.com/s/f0ewee5b1rc1bnw/beatthatflightLHS....
[1] https://www.dropbox.com/s/pxdvr2o8o96f6x7/beatthatflightRHS....
I've found a strategy I think believe will work -- my Leadership and the Environment podcast.
Here's the podcast: http://joshuaspodek.com/podcast
Here's an episode clarifying my strategy: https://shows.acast.com/leadership-and-the-environment/episo...
Here's my corporate strategy: https://shows.acast.com/leadership-and-the-environment/episo...
This sounds great, can't wait to check it out. After taking a Permaculture Design Course, I fell in love with the ideal of living a better, more "luxurious" life that has not only a lower impact on the planet, but a positive impact. I'm on my way down that path, but still figuring out how to make it replicable.
Working in my spare time on a command line terminal UI application that searches over source code and ranks the results.
It came about from watching a work college constantly opening VSCode when trying to find things in a codebase. I mentioned he should use ripgrep/silver searcher which he tried, but said he preferred to get more context and wanted ranked results. The context was possible using -A and -B but he didn't want that.
I had always wanted to make a terminal application and it seemed like an interesting problem to solve. I had also always wanted to implement BM25/TFIDF ranking algorithms myself and I was curious to see how well this could be done without pre-flighting and building an index.
Still a work in progress https://github.com/boyter/cs but coming along. Its usable now (with bugs) and is being used by my work mate.
I want it fast, but absolute performance is not a goal as its a trade off with any ranking algorithm. Not really the same sort of tool.
Anyway here is a search over my projects directory which is about 5.7 GB of content https://github.com/boyter/cs/blob/master/large-search.gif
Feel free to try it out if you are able to build from source. It certainly has some bugs in there over large codebases in the UI which I am aware of but does work generally.
The tui is indeed the sticking point and the thing people like the most. I had planned on sticking an embedded HTML page in there as well but that might be a bit much for a first version.
Is windows a supported platform?
EDIT I tried and no issues on Windows. I would need more details to investigate. Please raise the issue on github and I will investigate.
Also, working on how to integrate a small team of hackers into a big team of production oriented engineers. Making the first of something is such a different skill set to making thousands more.
I got here by getting headhunted for a neat-sounding job after a project elsewhere ended, and then assuming more and more duties until my title had to change to match my responsibilities.
Also I bet you’re excited for starlink.
Is this related to the refractivity of the near surface atmosphere above the waters surface affecting directionality and bandwidth?
Also the usual stuff. Hitting the gym (30 min a day, 5x a week), clearing out junk I don't need anymore, multivitamin, etc. 2020 is going to be the year of wellness for me.
EDIT: Forgot to mention this, DELETE YOUR SOCIAL MEDIA APPS. All of them. Use the mobile websites if you need to read them. Not having the icons on my home screen or app drawer made all the difference and really helped fix my cyberaddiction.
Come outside, it's wonderful.
I recently learned that homelessness is not just about the people you see on the street every day, but that homelessness is in fact a funnel that people fall deeper into as their situation becomes increasingly desperate. At the bottom of the funnel are the aforementioned group known as the "chronically homeless". The top of the funnel however, looks a lot different, it consists of people who might be couch surfing with friends, sleeping in cars or moving between motels. This group is known as the "hidden homeless". We likely encounter this group every day, at work, in the coffee shop, at the gym, but they look just like you and I so we fail to recognise their situation.
The "hidden homeless", at the top of the funnel, actually make up the vast majority of the homeless population. What's even more surprising is that this group overwhelmingly has access to technology, 90% have access to a smartphone or laptop with internet access.
The not-for-profit organisation I am involved with called Ample Labs (www.amplelabs.co) is working on developing chatbots to more rapidly connect this group with essential services. This allows us to get a better understanding of their behaviours, what services they use and how effective they are. This has two benefits - first by connecting the 'hidden homeless' with essential services quickly, we make it less likely that they will fall further down the funnel into chronic homelessness; secondly, it provides us with essential data that we share with cities to inform policy making.
The long term hope, is that by using data to prevent at-risk populations from falling deeper into homelessness we can combat the problem at its source and start to eliminate homelessness before it even begins.
Homelessness is a topic I know embarrassingly little about. What has the data led you to recommend to cities, policy-wise?
It seems like large-scale data collection should be (but isn't) used to inform many public policy decisions. Another area where I think data could have a huge impact is studying how the punishments that criminals are served in court affects their outcomes later in life, and requiring judges to factor that data into their verdicts.
Any cities south of the border?
Of course, there are plenty of national and state-level policy organizations; some even dip their toes in the municipal policy scene. But in the cities, most gropus are self-interested or focused on a single issue.
We're trying to fill the gap with original research and projects that operationalize the research of others -- taking, for example, good research and popularizing it, developing components for model ordinances, etc.
This is the first time I've heard of doing research with the express goal of creating actionable plans, but now that you've brought it up, it seems like that's exactly how policy-focused research should be.
There's a ton of problems when you're dealing with food though. Calculating calories of a recipe you find online can be tough. On one side, it's a natural language problem to extract the ingredient, the amount, the unit, and the prep/notes. On the other side, it's a data/data matching problem, where you need good data on a ton of ingredients, and then need to pick a reasonable one for "1 cup of milk".
And of course everyone eats and prepares food so differently that suggesting meals they'll actually enjoy is hard without asking them a bunch of questions first.
Worst case, I developed an app I like using more than MyFitnessPal just for me and I'll still be quite happy.
So many of the other tools are just unrealistic for how I cook, which is cook like 4-5 times a week max and never for breakfast or lunch (unless it's ahead of time).
I started this because I'm learning guitar mostly from YouTube, and I find myself constantly seeking videos to specific sections.
I'll probably launch the site on ShowHN soon. Feel free to DM me if you can think of other uses for this, or if you're interested to know when this launches.
Mostly I agree on hating things requiring a login but I'm okay with it in this case.
Any other insights you’ve discovered? I tracked my data for a year and discovered strawberries were a potent mood enhancer.
A centralized place to record things like that: - Arts: cinema, theater, books, ... - Travels: from, to, flight? car? - etc...
Started writing some code but let it for other things, maybe I would come back.
That's fun.
The new system is a quiet simple SOA arch with a dull, only-real data db layer, backend in Go with code-generation, frontend in es6 migrating to elm.
That looks the IT guys have when we say 'no really, we don't need IIS or Java', its priceless :)
The interesting part actually lies in handling both product management and sales for the new version while handling the day-to-day coding part.
Sometimes I think I should write a book on those subjects :)
My prediction is that it will be 10-20k lines of code when I'm done because there's so much obsolete cruft to remove. Plus ~1k of C as a shim layer to allow an incremental transition.
In this case I am avoiding the 'throw it all away and start from scratch' approach. It would be infeasible for the intervening period. I am putting together an approach that would get us there in a year or so, but we can lop off smaller chunks to rewrite (the existing architecture is a series of daemons, which helps us there).
1. (Assuming you're an outside contractor) How much do you charge?
2. How do you do estimates on this project? Or do you just do it bit by bit?
3. Are you switching over part by part or will deploy in a period?
If you can, you should at least write a blog post. A book will be amazing though!
At the beginning there was only vague goals, not estimates, now 4y later I can tell how much days a functionality will take within a 30% margin error. But it's only based on personal appreciation, tightly coupled to having personally designed the new arch from the start.
The migration strategy that I chose was to hijack the sales activity and pick which new projects should go on the new version. That way we can stay away from replacing years of specific functionalities for picky customers until we have a good idea of how much the business module is ready for prime-time.
I should definitely write a book, there's so much to say - maybe in 2020! :)
Short list:
* Pollution and the climate
* Privation
* Avoidable death
* Interplanetary settlement
* Liberty and communication
* Transportation
My primary focus is developing and commercializing reliable clean energy, because I believe that is the most effective way to further progress in the majority of the above problems.
To that extent, I've come to terms with an inability to spread my focus across all of them simultaneously and drive great results so instead I've taken an approach of working on a few of them full time myself and investing in efforts that work on others. My intent is to keep ~100% of my net worth invested in these main problems (either in my own or somebody else's projects) in perpetuity.
In my personal life, I've also recently been spending a lot of time thinking about health and purpose: how to build discipline, how people can/should decide what to do with their life, how to stay healthy and built fitness, etc.
Side project: in my free time over the last few weeks I've also been thinking more about how to create lasting models for information and media, and so I'm building a markup language / static site generator in that pursuit [2]
I particularly like your idea of keeping your entire net worth invested in the problems you're focused on, and I'm curious to hear more about it.
Also, consider that "keep ~100% of my net worth invested in these main problems" may not be the better strategy for funding those problems, versus "put net worth in growth investments" that can fund the same problems later on.
I don't believe in profiting from life-saving medication or anything like that, so my intention is to drive rapid growth with my activities in the energy, software, and automotive sectors to fund the less immediately profitable goals.
Make money solving the problems I can first, and later use that money to fund the further-off ones.
Currently this strategy is working well, as my assets are growing at above 2000% annually.
currently tracking is limited either by A) type of transportation (ships, rail, trucks) or B) by the Freight Forwarding company.
If you use multiple freight forwarders, you're stuck entering data from PDFs into spreadsheets to create your own custom usable dashboard.
If you use one freight forwarder, you have access to the main journey points, either as a spreadsheet, or if they're more sophisticated, through a web app. But I've only found one (silicon valley backed) Freight Forwarder [0] that gives the last-mile data -- e.g. last free pickup dates, pickup numbers, last free dropoff dates, return locations etc. -- through their web app.
This is critical for managing warehouse operations, especially for companies that handle their own last-mile (like we do), and it's been an absolute pain as we've scaled.
- a jdbc driver for interacting with google sheets
- a cross OS application which lets you share easily data
The first one is almost done and requires mostly documentation and some clean-up. It supports at the moment simple SQL queries like select * from, insert into foo() Values () and an update where I currently not remember the syntax. It also has already a Datagrip integration.
The second project shall work wireless and with minimal setup. The original idea was that devices search each other in the local network (via broadcast) and connect then. Further ideas which rised while development where:
- play sound on another device (which I initially thought would be super easy but seems like it is not)
- provide a possibility to define outside applications (like you provide a configuration file how I communicate with your application and this lets you show information on other devices)
- Not just device-to-device but also something like groups
- messaging with other devices
- more communication possibilites, ie via (outside) IP or Wi-Fi direct
Trying to fix uploading through tus.io (low level protocol) and uppy.io (user interface). Both open source and free to add to any project.
Were you being literal when you gave the examples of chemistry as hexation and biology as heptation? If so, why are they those levels specifically? Or were you just using those as examples because they're roughly one "step" apart on the hierarchy (i.e., molecules -> cells)? Sorry if this is a dumb question.
A humongous problem is the absolute lack of data that many psychedelic assisted therapists, guides, spiritualists have, to be able to point to their specific types of therapy as effective.
You come across folks that make humongous claims about the specific modalities they use, but don't track the progress of their patients and therefore don't have the data to prove it. We're working with volunteers at Tabularasa.ventures to develop some simple applications to both screen clients and also allow for practitioners or individuals to record progress (reductions in depression, PTSD, etc.) over time whether treating with microdosing, self administered, or more standard psychedelic assisted therapy (PAT) methods.
Happy to collaborate -> marik@tabularasa.ventures
We're basically trying to make an opt in service that can make procuring these relatively painless by grouping all relevant parties and then keeping these on record. A glorified KYC of sorts and then looking to use these as means of authenticating (I should be able to use my profile to sign up anywhere or transfer to my data (or parts thereof) to another party. Lots more to flush out but we have a good grasp of where we're going and what we want to achieve. Our government has tried to do this in the past but failed at getting it past the courts due to privacy concerns and are set to try again. I skimped on some details but the idea should be clear.
As well as new data protection laws introduced/proposed with more amendments to come, it's a simple but interesting problem at this point in time in navigating everything including how we do our own verification, security and eventual licensing to achieve the desired outcome.
Wouldn’t that just make a juicer target since a hacker would have one place to go?
I notice a lot of people help each other against the perceived government inflexibility so they’ll lend ID cards etc. won’t this just increase the amount of identifiers leaked when they do so?
https://github.com/moj-analytical-services/sparklink
It's currently in alpha testing, but the goal is for it to:
- Work at much greater scale than current open source implementations (the ambition is to work at 100 million records plus)
- Get results much faster than current open source implementations - ideally runtimes of less than an hour.
- Have a highly transparent methodology, so the match scores can be easily explained both graphically and in words (it isn't a 'black box')
- Have accuracy similar to the best products on the marketplace.
I arrived at this idea from two directions. The first direction is that I sometimes try to code review some of the questions over at CodeReview SE, and the whole thing feels unergonomic. I dislike scrolling up and down to check the code and constantly losing track of the things I'm reviewing. This is where I think inline commenting would help. Also, there is not a lot of room for discussion. You only get those comments below the review, where you only have a few characters to argue your point. The second direction is that I produce code snippets (programming homework, short snippets at work, etc.) which I would like to submit for review. I don't always want to submit it to the entire internet for review. I just want to get a private link to the code review, which I can share with my colleagues so they can review it. Kind of like a reviewable PasteBin.
Some of the features I would like to add: importing files from GitHub for reviewing and users could import their unanswered CodeReview SE questions for another review.
I really dislike CodeReview SE for many reasons though, I don't think this Q&A format is suitable for doing CRs
It's been fun to work around the constraints on an underpowered device. It's also an excuse to learn ARM assembly, and a nice break from all the JavaScript I've been spending my time in lately!
There are many research papers talking about it for many years, but till recently there was not cheap enough hardware available so it just get stuck in university laboratories.
It is still tough project to pull out as it combine hardware, cloud software, machine learning and there is quite some laboratory work required as well. Doing all of this as a single person and bootstraping is extra challenge but I guess I don't know better.
I've got into it 5 years ago, when I decided to quit technology, bought small farm and build winery. At first I wanted to analyze the wine itself, basically to make traditional method obsolete, but performance of this kind of instruments are not good enough for liquid that complex. It turned out grape analysis is much easier target to tackle.
The hassle of splitting proceeds from a service/event/product sale after the fact related to sending/collecting your % share, timing and details of wiring the proceeds.
Solution:
Pre-set allocations and create a customized checkout so that splits happen on a per-payment basis. Members dont have to wait to get their share.
Idea kind of came about after watching my wife, who is a yoga instructor/ studio owner try to split proceeds from a workshop she hosted with a few collaborators.
Another example: allows you to create a shield to a checkout that will split proceeds on a per payment basis.
https://github.com/surfertas/deep_learning/tree/master/proje...
Working on this on my spare time. Any advice from the community would be greatly appreciated.
Basically just leveraging a lot of Stripes features which are great. Their support has been super helpful as well advising on what can and can not be done.
Noted on the onboarding issue. Will check whats going on there. When I signed up myself, gmail classified it as junk/spam, possibly rightfully so.
Thanks again.
http://aperocky.com/prehistoric
It's already got a pretty sophisticated production logic, and also a unified market.
Looking to add a few functionality like child support, new resource types and maybe eventually a governmental system. Can even try out different government strategies.
If you have any ideas please share. It's been my passion for 2020 so far.
Being a good hacker, I pulled at that thread until I had another, and another. Now I'm writing about semiotics, language, lambda calculus, and philosophy of science stuff. It's all related to my original quest for a better explanation, and it affects everything from AI to coding practices. I'm about done now. Now the trick will be getting it all in a format that's consumable by the average programmer.
It's definitely one of the most interesting projects I've ever worked on!
Having run a cybersecurity services business for three years and previously working for federal clients, I know that government and large banks are sucking the talent up, leaving fintechs two options: ignore security or overpay.
On the reverse side, there are lots of talented independent providers who simply need somebody to vouch for their skills. We meet with and vet everybody on our platform to make sure they have the capabilities.
Will be launching a prototype to replace this landing page shortly. If you're in the New York area and are either looking for cybersecurity contractors or looking for a project, I would love to get your input!
Most companies at one point are internally not aligned, marketing fighting product fighting development fighting design fighting sales.
All are wanting to contribute value, all hindering each other in the process of doing so.
The goal is one framework where a) an initiative can start from any group/team/individual within the company b) every other part of the company can rally behind - with their own expertise and point of view.
I always start with a talk (gave the first about it last Thursday https://jtbd.ws/), then I take it from there.
We've had a ton of great feedback from customers, and we are working on several new sleep technologies that we plan to release this year.
It's also been interesting to apply the lean methodology to hardware. Iteration cycles are long, but I'd argue that lean is just as important for hardware as it is for software.
https://www.ies.org/fires/circadian-lighting-an-engineers-pe...
- The metrics Circadian Stimulus and Melanopic Lux are flawed
- We don't know enough to produce products yet
I agree with him on point 1, especially about CS. The author, and several others, have demonstrated a huge discontinuity around 3500K. The CS model needs improvement.
Regarding melanopic lux, the author says it is more a problem with our measurement and modeling tools. Most manufacturers won't provide a Spectral Power Distribution, but this is starting to change (we do). There is at least one clinical trial in progress that is testing this metric directly, and several experts think it shows a lot of promise.
With that said, new research on blue vs. yellow/orange opponency is coming out. Basically, we're trying to figure out contribution of the visual system on top of melanopsin. I'm not sure what to make of it yet, but I think we'll find out much more in the next 2 years. We probably need a better metric than melanopic lux.
On point 2, I agree that our knowledge is changing. But it takes a number of years for standards to be developed, much slower than the rate of technology improvement. We are paying attention to all the new research, but we based our design on our best understanding of the research to date.
We know that we can really only attract early adopters at this stage who "get" it, and that most people are going to wait and see until things are more standardized. But we are effectively able to do mini-experiments with our customers, which can lead to more insights/avenues of experimentation.
Case-in-point: f.lux. Research has shown that it is somewhat effective on its own, but combined with dimming, it could be quite effective. That's actually what we're doing with our product—controlling both the spectrum and intensity. And f.lux is able to run experiments with a portion of their users to advance our understanding.
Basically, my opinion is that we have to start somewhere, and we believe we have enough evidence supporting that our solution is "better than nothing." We were able to prove that there is enough of a market for these types of solutions already, even though our knowledge is still evolving.
(edited for a paragraph spacing typo and a 2nd time for a grammar issue)
You may think extremely bright SAD lights is already readily available, but there's anecdotal evidence to suggest they are insufficient
https://www.lesswrong.com/posts/koRZu53LBZEapwww6/could-some...
Big core temperature changes usually make me feel sleepy.
We're at a very early stage and looking for investors.
Regarding regulations we are studying two possible scenarios, comply with all or simply Uberize the model and let people sell money for a fee. For the former we need tons of money, attorneys and on/off ramps with the banking world. For the latter we only would be the messaging transport and people would buy and sell money informally, that's the long tail of the unbanked and informal merchants.
I just like reimagining things, trying to elucidate first principles and go from there.
I'm also interested to hear what you think are the shortcomings or limitations of our current idea of a (smart)phone interface.
As we connect classrooms and scale across different countries, the problem set has grown exponentially.
There is an unlisted sample video in there that I've put out for feedback, and I'm making changes based on that. Will also be putting together related content around GCP.
p.s. do subscribe to the channel.
One reason I'm targeting parsers in particular is because I've been finding a lot of modern programming language books are a bit anti-parsing these days. EOPL avoids parsing altogether by using a parser generator, effectively saying that it's a hard problem. PLAI outright calls parsing a "distraction". SICP (not strictly a compiler book, I know) and Lisp in Small Pieces just use the triviality of parsing () languages, which I feel doesn't generalize well.
I emailed the author of PLAI (Shriram Krishnamurthi) about this. His response was effectively that modern books come off anti-parsing as a reaction to old books, which were parser heavy, and tools like YACC -- "Yet Another Compiler Constructor" -- even though it's just a parser generator, not a compiler constructor! He went further to say that, given parsing is roughly trivial in () languages, it sort of seems parsing is only incidentally a compiler/interpreter problem, and users of () languages view non-trivial parsing as signalling a design flaw. I found this to be an interesting take, but in my day job I generally don't have much say in the design of "languages" of semi-structured text that gets thrown my way.
Anyway, I know the Dragon Book covers parsing in some detail but for some reason it's been kind of impenetrable for me -- it feels a bit more abstract than I like. I can follow it, but while reading it I can't help but wonder -- "is this actually going to help me in practice?"
I recently have been reading Niklaus Wirth's stuff though, like the last chapter in his algorithms book and his Compiler Construction book, and those have been absolutely fantastic.
I also asked a question on SE about a particular parser I'm working on -- if anyone has some thoughts I'd love to hear them :)
https://codereview.stackexchange.com/questions/236222/recurs...
https://www.whoi.edu/press-room/news-release/whoi-awarded-1-...
Additionally I just won a grant at work to begin designing and building an open source underwater glider. Underwater gliders are one of the best ways of carrying instruments to sample the ocean. They can last 6+ months and be directed to interesting area's. The billion dollar companies that make and sell underwater gliders are focused on oil+gas+military business and are not giving the service, support or product depth the science community needs. They are in dire need of a tech refresh - they fail a lot for an old technology and run DOS. The only way we have a chance of understanding the ocean is to make sampling the ocean more affordable, reliable and accessible.
I thought of doing something similar but based on a single vertical line, it probably needs to be flat to account for flows and thermal gradients refracting sound.
What cpus does it use? Is there a local backup of the sound field on the buoy? Is the full sound field sent up, or does it have to be "pointed at" something?
I’m starting with the sharing of common information with clients and partners. Organizations are often required to supply information on a regular basis to a wide range of clients and partners (bank account details, company, company registration details, tax clearance documents, certifications, charity registration number, etc.). A lot of these documents need to be renewed on an annual basis so there is a constant stream of requests for updated versions.
For bank accounts, the ability to verify a bank account automatically can prevent invoice fraud.
I’m looking at a model where a piece of business information is uploaded to a central platform and then provide permission for others to access it and to receive notifications when a new version is available.
In the first startup school batch for 2020 and working on validating the problem with actual users.
From our experience the biggest issue students have is, they can't solve an issue because they didn't understand a concept they have already "learned" in the past. It's simple, yet powerful.
But, we are still a very small start-up.
At the Moment, I'm fighting with a monolithic, untouched Java 8 / JavaEE6 service which has lots of old dependencies and that uses old cryptographic ciphers, some of them classified as unsafe (e.g. brainpool512p1).
None knows how to make a reproducible build, since everyone gets a different and working or not-working package and some modules are not even released (using the infamous -SNAPSHOT) in maven and there's no documentation. Unfortunately, there's little testing, so everything can be broken easily and none can know it.
Some developers are also really undisciplined, touching code but not running end-to-end (manual) testing, not even running the installer.
If I had the decision power, I would throw this thing away and start from scratch, probably without Java too or, if Java, at least the latest one and maybe Spring, not JavaEE: Wildfly moves too fast and each release breaks compatibility with the previous one, concerning settings (RedHat: why do you do this??)
As a fun project, as I already had code to generate llvm bitcode from .NET, I now do mem2reg (convert stack spots to SSA registers), dead code elimination, constant folding and other small optimizations. That part now works, and I managed to create a simple x86_64 coff object file (with everything needed to link to it, including pdata, xdata) that returns the "max" value for a given integer.
That is about all that works for now, and I don't get to spend much time on it, but the end goal is to have a "good enough" codegenerator for non optimized cases, that could potentially be faster than llvm (to emit). The primary goal is to learn how to do this though :)
Having previously worked at a marketing company and a startup, it’s been fascinating to experience a legacy manufacturing company growing (or trying to grow) into the future.
Yes, the engineering problems are fun and all, but I think the most fascinating part has been thinking about what American manufacturing will look like 5, 10, 20 years down the road.
In my experiences, I believe American manufacturers will NEED to invest in industry 4.0 tech in order to mitigate costs associated with rising wages, shortages of skilled machinist labor, and greater demand from consumers/regulators/OEMs for information and transparency.
I’ve also been quite amazed at how much paper is still used and the lack of industrial software products with quality UX.
And I don’t think American manufacturing will ever cease to exist.
I have quite a few domain names that I have purchased over the years that I am not doing anything with at the moment.
I wanted minimal amount of work to make a good use out of these domains.
So I built Newsy. It turns your idle domain into a news aggregator.
I’m nearly there. You can sign up and I’ll invite you to check it out!
Hard to think about it just in my mind with no visuals.
IMO the current options are too complicated or expensive and appropriate for the largest companies. I cannot hack a simple application for data discovery or usage statistics. So I am building a dead simple data catalog that I can reuse. The data lineage app is the first app on it.
(1) https://github.com/tokern/piicatcher (2) https://github.com/tokern/lineage
Launching soon.
Yes, there are ways to be clever about this stuff, but for "real music making" the typical size of the data involved makes cloud-based collaboration less immediately appealling than you might imagine.
Just checked out Ardour, looks great! Being able to work with videos and (I presume) work on additional audio and eventually mix the two audio sources back into the video is a fantastic feature.
Explaining the implementation would take a wall of text, just listed the aspects that make it interesting.
During a sabbatical trip in the Canadian Arctic in 1975, I came in close proximity with a beluga in Hudson Bay and was impressed with the unusual vocalizations which was in-air and about 3 feet from me. The beluga was tragically killed by Inuit a few minutes later. That's how my interest started. I later learned the basics from two people who were leaders in this field.
Hopefully in 2020 this means more than simply resurrecting a clone of MS Office's Clippy...
Increasing food safety and security plus availability and choice in urban environments using robotics to automate food preparation and software to manage operations and logistics. Hopefully also make money. Differences from web stuff: includes embedded, mobile, electronics design, mechanical design, fabrication, business, cross-border operations, food safety regulations, etc.
I'm interested in what value we place on connecting people in our network. Say you meet two persons in your travels that might benefit from being connected. What value do you place on making this connection? How does it make you feel? Does it have to have an obvious personal utility?
These are the types of questions I would love to investigate further. Maybe they're kinder, calmer tools that we can build for this specific purpose.
When signing up for services that require real identities (banking, insurance, etc.) the standard currently is to require a picture of a passport, a video of yourself, or copies of some paperwork. These methods are all high-friction and provide dubious security and privacy. This is already a solved problem in some countries and I'm working on the equivalent on a larger scale, without the geographical restrictions.
If there is anybody else here working in this space then feel free to reach out!
This is a great problem and I'd love to know more.
I'm from Sweden where basically every single person has an app on their phone which functions as a digital ID, used by the government and a ton of companies. Every time we sign in to (or sign up for) services that handle taxes, banking, insurance, benefits, payments, utilities, etc. we use our mobile ID. There are a few other countries that I know have similar solutions, for example Finland and Estonia but many still don't. You can read more about it here: https://en.wikipedia.org/wiki/Mobile_signature
The big problem is that these solutions are all different and it's close to impossible to integrate them all. This is the reason why many global startups that require identity verification fall back to less convenient and secure methods, e.g. Revolut, Monzo and N26, to name a few. I believe that for truly convenient identity online there needs to be a single digital ID which works for everyone. This is what I'm currently working to achieve.
If you have any thoughts I'd love to discuss it further!
There are many opportunities that could really use a good mechanism to uniquely identify users, know for sure they are real people. This is pretty hard to do (or outright impossible) without the collaboration of governments. But governments will f*ck it up in a number of ways if we leave it to them. We have to think bigger than what some countries have already solved.
First, we need something like credit cards. A physical object (identity cards could work on many countries, but they tend to suffer from beyond horrible usability when it comes to their digital chips / functionality) with a password that can be changed. We would also need a place to see all the "transactions" or actions done with your identity, as we have with credit cards. India and their Aadhaar project has shown that biometric identification is not good. But it sounds very nice and sci-fi, so it sadly sticks. Nothing new yet.
But what we really would need are manageable permissions, so you can always prove that you are an actual human, but not necessarily revealing anything else and being a completely anonymous user, or choosing to reveal some data (country, real name, etc).
If something like this was global and effective, not only we would have many more opportunities through the internet, we could also come much closer to things like direct democracy. Password management and online identity management would also become much easier. Obviously there also are many problems. Starting with the access to the internet itself. And all this identification system does sound very dangerous from a privacy-minded perspective (but the alternatives will end up being much worse, and worse systems will be imposed on us). And I'm still completely ignoring the political will to do something like this.
Privacy is definitely at the center of what I'm building. My approach is to put full control of their own data in the users hands. Data is only shared with a service when a user explicitly allows it and the user is always aware of what that data is. Your idea of being able to share nothing at all is also something I've been thinking a lot about. Being able to verify an identity without leaking any PII is one of my main goals.
I also agree with you on the issue of governments. There are very few who have managed to introduce a digital ID locally. A large amount of countries coming together and building a common solution seems very far-fetched currently. Where I'm from, Sweden, we have a well functioning digital ID used by everybody. Funny thing is that it was created by the private banks and only later adopted by the government.
It's definitely a hard problem but judging by the evidence it's solvable, just hope I'm on the right path. If you have any more thoughts I'd love to hear them!
Have your real world identity issuing agency (whatever that is in a specific country) become a CA, issue personal cert and store it in crypto-hardware. Smart-card interface is rarely present (generally requires specific hardware), USB requires drivers.
Heck, the key can even live inside your SIM card if you prove your identity to mobile provider, which you have to if it is a contract anyway. Works well and is easy to use. Can be an attack vector for scams, though. https://en.wikipedia.org/wiki/Mobile_signature
The problem lies with the issuing and trust. Many countries do not issue e-IDs and it's not easy for a business to support all the ones that do. Take for example my country, Sweden. Basically every single citizen has an e-ID and it's used for everything. Yet I can't use it if I want to sign up for a non-Swedish service such as Revolut, N26, Transferwise etc.
- a graph-based task manager that incorporates dependencies between tasks and infinitely-nested subtasks - IE maps to how we actually think about tasks being related and broken down. Aiming to get this one shared with the world in early Feb.
- a visual programming environment that represents how we model software in our heads, not how it runs on the computer/s. This is my longer-term, much more experimental project.
Drop me an email (in my bio) if you're interested in either! I'll be commercialising the former quite soon and I'm putting a lot of effort into pleasant to use.
https://github.com/ikorb/gcvideo
GCvideo has a way to convert the digital signal on the N64 into composite video, and has VHDL to create an HDMI signal With Audio. So I have been working on finding the digital Audio out on the N64, and converting the whole signal to HDMI.
In not so many words, I am recreating this from scratch: https://www.retrorgb.com/ultrahdmi.html
Mainly because it is impossible to find that.
I built a similar tool internally at my last company and we used it to alert on things like employees making google drive files public to the internet, okta configuration changes, github ssh deploy keys getting added, employees logging in from foreign countries, etc.
If anyone wants to check it out you can reach me at arkadiy{at}logsnitch.com (or just sign up at the same domain).
Been a really tough journey. I’m was the only coder and designer in the project for the longest time, and my development skills weren’t really that good when I started building this.
Here’s a link to it https://nyxo.app
Nice start! Is there any fitness trackers you believe outperform in terms of sleep tracking? It's an interesting idea for an app. because I have personally found specialists are booked 3+ months in advance.
Detecting when person is asleep has become quite good, but there's still a lot of work to do in also detecting sleep stages. I would not trust any wearables deep sleep readings too much.
There are some interesting value-judgements that have to be made here (e.g. do we value the consumption of future generations more/less than present consumption?), so I suspect there will never be an objective answer to this question.
The model was only a little over 70% accurate, but the data was sloppy because we had to get creative mapping precincts to districts since their boundaries don't overlap exactly. However, I think this could be significantly improved since crime data is geo-tagged, so you can get much more specific about where crimes occur with respect to funded projects.
I think that model can be made to help inform where funding could be most beneficial from a social perspective. I realize there are other factors in infrastructure investments than crime - I was focussed on community improvement projects (e.g. blight removal, green space development, school construction/repair, etc.)
It's currently in working state. https://raw.githubusercontent.com/loa-in-/python3-dreamwork/...
You can see current output here: https://github.com/loa-in-/python3-dreamwork/tree/testout/te...
Funny thing is, the basic software more or less exists already. At least on the fulfillment and logistics side of things. The tricky thing now is to create the physical network (also companies like DHL ship for everyone, even next day) and come up with the processes to match n retailers with m dropshippers (some of them shared between retailers) and a basically infinite amount of consumers.
I said long shot. First step is to get my 4PL company of the ground. A 4PL is a nice first step, I tke care of daily logistic operations for clients. Which also includes pretty early on a Dropship component. So once the 4PL is earning some money, the next step will be to define processes for a scaleable Dropship solution, identify software gaps and then create the platform. Talking about longshots...
How I got the idea? I worked for Amazon running, among other things the Amazon.de dropship network. After that I worked for a producer of solar modules. That company sld some of the modules through a webshop and had some dropshipping. Totally inefficient, intransparent and expensive. So I told myself, that can be done better. Took me three years to take the leap into startup world.
there's a lot of "greenwashing" in the industry driven by opaqueness and lack of measurable data.
step #1 is to get more brands on board. step #2 make it easier to monitor supply-chain and have actionable and measurable KPIs built around data.
That being said, Poshmark does have a $600m valuation, so there is a market here.
I've been reading about this for years and recently started sending out short summaries of what I've learned (typically geared at how the lessons can by applied practically).
Last week I shared how Nobel laureates are 22 times more likely to have a side hobby as a performer than their peers.
Ultimately, I am trying to land on a succinct answer to "how do you channel broad interests and talents into an impactful career?"
(this is my email: https://stewfortier.com/subscribe)
I too have really broad interests...I find basically everything interesting, which is both a blessing and a curse, as I'm sure you've experienced. I'd love to talk about this more...do you feel you've come to any kind of answer on how to focus your wide interests?
One theme I'm starting to converge on is the idea of a generalist as an "expert" at a) maintaining a wide range of mental models and skillsets and b) developing a sense of which type of problems to apply each to.
In other words, effective generalists become good at knowing which speciality or approach should be applied to a problem, even if they "only" grasp the basics of any one discipline.
Example:
A software engineer wants to develop deeper friendships. They may think that building an app that reminds them to keep in touch with friends will help.
Of course they think that... software is what they know best.
But a generalist may take a different angle and see that the root cause isn't an automation / information problem, it may be a human psychology issue.
"The real problem is that you don't believe you're worthy of love. If you work on that, you may feel confident enough to want to reach out more."
The next email is going to start outlining the most practical, effective mini-mental models that generalists can use to solve practical problems.
That's a really interesting, but sensible, conclusion to come to. It seems to follow that generalists would make great business/personal coaches, as they're good at pointing people in the right direction. I'd be curious to look at great coaches and see if they had a ton of different interests.
I'm stoked for the next email :)
It's also somewhat of a relief to read.
I think intuitively many people feel that range matters, but fear that we'll sound like we lack a "speciality" or even "hard skills" if we proclaim ourselves generalists or broadly curious.
2. At work, I recently completed a really long project with a large team. I'm trying to make the lessons learned accessible to others in the company because they'll also be undertaking similar projects soon. That means documenting my learnings at a level of abstraction that allows others to not make the same mistakes as us, but still have enough flexibility to tailor their implementation based on their team's needs. The hard part is the intersection of technical and people-oriented knowledge dissemination.
This year is going to be focused on a lot of teaching, which I'm excited about.
https://datasetsearch.research.google.com/search?query=whole...
Teaching machines to diagnose cancer with superhuman sensitivity and specificity makes it easy to sleep at night.
Currently, I don't know any way how to initiate execution of scripts over http server so there is a systemd timer checking changes in files and recompiling the whole site. This has lot of downsides. The easiest would be if the static generator reacts on existence of a specific file - recompiles the site and removes the file afterwards.
You call an endpoint anywhere on the planet and give the name of the service you want, which then gives you access to that service's published API (similar to how you'd use import and gain access to a library's API).
To start, it will operate over port 80/443 to allow seamless integration into the current world infrastructure, but I'm also hoping that in maybe 10 years it could replace HTTPS entirely, possibly even TCP.
The first step is an encoding mechanism that supports the most common data types natively, which I've defined here [1], and am currently writing implementations for in go. It's a parallel text and binary encoding so that we don't waste so much time generating bloated text that's just going to be machine-parsed at the other end, but also allows converting on-demand to a text format that humans can read/write. I ended up developing new encoding schemes for floating point values [2] and dates [3] to use in the binary format.
The next layer above that is a generic streaming protocol [4], which can operate on top of anything from i2c to full-on HTTP(S), and supports encryption. It's designed to be as non-chatty as possible so that for many applications, you simply open the connection and start talking without even waiting for the other side's acknowledgement. It supports bi-directional asynchronous message sending with chunking and optional acknowledgement on a per-message basis, with tuneable, negotiable message header size.
The final layer will be the RPC implementation itself. I want this as a thin layer on top of streamux because many of the projects I have in mind don't need full-on RPC. This part is still only in my head, but if I've designed the lower layers correctly, it should be pretty thin.
[1] https://github.com/kstenerud/concise-encoding
[2] https://github.com/kstenerud/compact-float
You host a copy of my web application, and it handles all your user account stuff with modules that add organizations, Stripe Subscriptions and marketplaces powered by Stripe Connect. You write your application with its own web server in whatever language and the two servers form one site.
At the moment I am trying to finish automating my documentation based on the test suites including API details from API tests and screenshots from UI tests.
I am looking for testers if you are building a SaaS or a Connect marketplace.
Whenever we begin to do something, our computer just sees a bunch of apps and windows. It never tells us how to get better or does things on our behalf. At Amna, we’re working on a natural interface structured around the way people think. We believe it will change the way you interact with computers, and the way computers learn from us.
full problem: https://getamna.com/blog/post/amna-solves-problems/
In 2020, we plan to continue the VR Roadshow and brainstorm new ideas to bring more awareness to virtual reality tech.
Professionally - change IoT into one big robot, make platform to connect ALL devices with one system, essentially what Bruce Schneier warns us about[0].
[0] https://www.schneier.com/blog/archives/2016/02/the_internet_...
Hence I started a website to curate cool & impact projects that people are building that nobody knows because they are small or unknown (yet). So kind of discover amazing companies / make impact kinda thing. Hoping to launch this month.
I have build a VERY basic landing page but I am struggling to get time to spend on it.
I have to say that the technical challenges of bringing in modern web technologies to interface with legacy systems has been an interesting (and frustrating at times!) experience. After working as a software dev for a number of years before taking this on, I’ve been jumping between sales, marketing, devops, management, and actual software development all in a day.
Because of this the systems end up become getting fragmented over time to handle all these different cases and specific needs for customers. Before I joined, the developers have attempted to unify as much as they could, but the business need wasn’t quite there to justify it as much as it should.
Building out our initial SaaS offerings have helped a lot for us since there’s the concept of only having one set of code running on an instance. Because of that, we’re able to abstract away all of the different nuances of each system such that the cloud servers don’t end up in a fragmented state as well, but leaving the on-premise legacy system as it is for now. The plan is for the core ERP to eventually move in that direction, but so far we’re chipping away all the edge add-ons and functions first, such as API integrations with Shopify, Amazon, etc. and building e-commerce storefronts for our customers in React/Node.js.
The implant helps patients perceive visual information about their surroundings.
Pretty cool tech and fun to work on, too.
So, for example, the input is 'nytimes.com' and the output will be the last headlines.
Plan to release it in a few weeks.
1. Heatmaps based on all popular gradient based explainable AI techniques (plus our own) for classification, regression and semantic segmentation tasks.
2. Uncertainty modeling for classification, segmentation and regression tasks.
3. Concept Discovery/ Pattern Discovery (and dependence) for patterns learned within a deep neural network. (Loosely based on TCAV)
4. Using network internals for optimal pruning and model compression.
Send us an email at sales@untangle.ai if you’re interested in trying out our toolkit. We offer 30 day free trial period.
It might not be as impressive as some of the comments, but it does seem like something the market is needing.
I'm also working as a contractor on automated valuation systems for real estate properties, mainly for the argentinian market. The company have already sold the service to a big international bank to periodically update their mortgages.
And now I'm pondering about starting a research+prototype AI consultancy.
https://github.com/ityonemo/zigler/
On my plate currently: Figure out how to make a long-running zig function run in an OS thread alongside the BEAM so that it doesn't have to be dirty-scheduled.
- how to do digital identity in health and public services for ~15m people
- replacing enterprise/waterfall security risk assessment with collaboration and iteration.
- applying product management methods in the public sector
https://www.instagram.com/starshiprobots
Technology and business do work, so we probably will have thousands of robots within a year, and millions not long after that )
The hardest part has been deciding what to fight first and meeting other people who have experience working with algae. I would love to connect with anyone that wants to talk Algae!
I create computer models of water networks and calibrate them so utilities can do what-if and growth scenario planning. (e.g. what happens if this pipe burst? how would the network cope with 20k new houses in 40+ years)
I'm also developing software to help water engineers build and run models, some of it opensource and some of it commercial.
I'm currently pushing most of my effort into an opensource javascript library to simulate water networks.
Since the type information is erased at compile time, it uses the compiler API to extract the data needed and generates TS code for the interface and constructor mappings.
The library is on GH, but not really much to show. I've posted on /r/node and it got some positive reactions, but it didn't got that much attention.
My fun stuff at the moment:
1. Learning Windows IoT on a Raspberry Pi 3B
2. Working on proof-of-concept Search Engine Indexers for specific datasets and/or local file-systems (on network servers).
3. Exploring a new paradigm of allowing people to easily publish train-of-thought type content without having to post a long series of tweets or silo it inside Facebook/LinkedIn/Gist etc.
Business listings are fairly sparse in some countries. Many owners do not bother creating even a google maps profile and just rely on word-of-mouth for new clients. Acquiring the bottom of the data-iceberg will require some creativity going forward.
Anyways, just sent you a mail.
The problem this solves is sharing. Sharing between devices/users should be as simple as copy/paste initiated by either user like everything is local.
But its more than just that. I want to take the material and make it more entertaining as well as educational.
I am observing that more and more kids are learning things on their own by just going online and searching for videos on how to do X. We are on the cusp of online learning overtaking traditional in classroom learning in terms of quality and presentation.
I'm happy that it works well for many small sized problems.
https://bustl-app.com - A SaaS product that acts as a personal assistant that will integrate with a range of different apps.
What I can't figure out is what to use as inputs, similarly to the human senses, so that it doesn't become too specific, i.e. weak, but instead remains general and able to understand the binary language computers use.
Predictions are a critical part of decision-making, and it's possible to improve – see, for example, Philip Tetlock's work. But that requires the right tools, which we are building: https://www.empiricast.com
I'm mostly heads down coding every day, building an MVP. Also trying to find some investor interest where possible, however fundraising has never been something I'm good at.
While this sounds like a complainy pants problem, this isa very real problem for a very large percentage of the United States. Without a 4-year degree as a de facto dues card, you are severely limited on your options.
At 34 years old, I could maybe have a degree by 40 while working full time and have to take on 30-60k dollars of debt to be competing for entry level jobs against 20-22 year old applicants (many schools now have programs so students can graduate simultaneously with a high school diploma and an associates degree). At my current income, if a degree could get me an extra 15% within a year of graduation, I would be in my 50s before I paid the loans off at current rates. That means I sacrifice the last half of my 30's to break even in my 50's and maybe make some extra money in my 50's and 60's losing out on 20-30 years of compounding interest because I don't have that arbitrary degree in anything as a dues card to say I'm worth hiring/promoting.
Blah.
Last year I made about 10% less than the year before because of zero overtime, our annual merit-based increases often are break even (sometimes not even break even) once you factor in inflation and insurance cost increases, throw in the constant nagging pressure of cancer risks (father died of it, mother had it, father's mother died of it), climate change, international trade issues which could see me laid off, automation possibly replacing jobs in the near future, it can often be quite crushing. Especially when you're trying to maintain sobriety and just want to run off into the woods with a cask of high proof alcohol and try and befriend a bigfoot to help provide food and shelter for you so you can die from Lyme disease or exposure living as a refugee in Bigfootville.
Meanwhile you see people with YouTube channels buying what equate to mansions (What's Inside, Jenna Marbles) and taking international trips monthly (What's Inside, Casey Neistat used to, on aircraft with seats in the tens of thousands of dollars a flight) and even crazy domestic trips frequently (What's Inside) and you're like, "Dude, I just want to make more than 34k a year".
I truly can't imagine what it is like for people that are consigned to working fast food/retail/service jobs as their sole source of income. It has to be all but crippling.
I've had too many friends and family members end up at companies that were not a match and watched the massive stress pile up. I want to help people find the right team/culture for them.
How do you differ from something like https://www.keyvalues.com/
We differ in a lot of ways, but the biggest is that we are trying to profile the actual eng team and how they work / what they value. And, then match you with teams that are a good fit + high satisfaction in key areas.
For example, say you are motivated by big tech challenges, we would match you with teams that are motivated by similar and report satisfaction in that area from their current eng team.
I am working on creating a solution that gathers the data normally not seen in console dashboard and discovering actionables that help the user.
It's an interesting problem, requiring both dexterous manipulation and long-term planning. It's also compositional, so I believe some form of hierarchical control and planning can solve it.
www.clvrai.com/furniture
2. An all-encompassing personal knowledge management solution that is effortless and universal.
Why am I still using pen and paper for math homework? Why do I have to rewrite the whole friggin thing every step?
And there's a hope that whatever I learn might be useful for lisps, too.
1. 4-manifold problem, while you can see it should be the surface volume of the shape is equal the math proof is the impossible rub. 2 Prime number generator
This can lead to ai-suggested interventions that people can apply to themselves or to support someone else.
It's a complex problem, but there must be a better way of doing things!
Sounds easy...
But it's the most difficult thing I've ever tackled. Even considering I've read books like water since I was a kid.
released recently as my first app in google play
https://play.google.com/store/apps/details?id=com.due.core&h...
It is a next generation carbon offset marketplace.
Like a poor man's version of this book by Microsoft CTO Nathan Myhrvold.
https://www.amazon.com/Modernist-Cuisine-Art-Science-Cooking...
We help you avoid distractions while working on websites and the internet by installing our Chrome extension. For example, Baitblock removes recommended videos on YouTube while you're working.
It also deals with 1st party cookie tracking. It clears cookie/storage on every page load as long as it detects that you're not logged in to the website (upcoming version removes many bugs) using machine learning (NLP).
Since there are too many cookie/gdpr popups now a days, Baitblock automatically hides them while you're working.
You can also add summaries/TL;DR for any link on a website (right click) so others dont have to click.
The end goal of Baitblock is to block all possible distractions in a webpage and save everyone's time.
The latest version of Baitblock 0.1.0 is awaiting approval with many fixes and new features.
Lots of webscraping.
Maybe we can compare notes?
My Open Collective page (I’m ballin’ on a slim budget here) https://opencollective.com/techfins
You can see more of my fins effort on my instagram page, @stormfins
I recently decided to use the techfins name instead.
I ended up working on this thanks to a passion for surfing, and knowledge that new airfoils could radically improve my surfing ability by augmenting my surfboards’ capabilities. I learned CAD four years ago just to do this - make fins based on new airfoil templates. This ‘new class’ is essentially high lift fins. Compared to the current surfboard fin standard (6mm thick fins), my 16mm thick fin designs provide radically more drive, traction and stability to surfing at the lower speeds. Making normal to pumping surf more accessible and enjoyable to novices and experts.
These fins are not only empowering for surfing ability, they’re also safer because of their thicker, more rounded edges, and when 3D printed the fact they break before your skin does. Also, if these fins could be fitted with an internal floatation during printing, they could be recovered and glued back into place using automotive plastic glue.
These are also literally the first high performance Wavestorm fins to be created as well. Anything else out there is a boring, simple fin design.
Some past feats I can be proud of that you all may appreciate:
• created BeelineReader.com’s first working app, helping them get off the ground. Helps you read much faster using a novel, internationally patented innovation.
• Created a web browser with T9Space.com that empowered 10s of thousands of Nokia phone users around the world to access the desktop only internet back in ‘07-‘10
Earned a bachelor’s in CE at UCSC 20 years ago.
My moniker thirdsurf is about a P2P ‘school of surfing’ project I want to get going next. There needs to be a more dynamic connection between those with the knowledge and those who’d like to learn.
If this fins thing gets off the ground, it’ll open up other possibilities. Nucleos.com is an example of a company that’s been developing a cheap school software server that can operate in developing world conditions. I want to see 3D printing fin labs sprouting that use a computer that can serve out edu apps to anyone nearby who has a wifi device.
I also aim to set up a P2P market for people who could fabricate the digitial fin designs I’ll be making available. This could open up a market of innovation, empowering people to tout novel materials and fabrication methods, helping advance greener, safer and more economical ways to make these fins, and other goods.
Also, in places where it may be difficult to obtain filament, the machines https://preciousplastic.com/ and others are developing could turn plastic refuse in devloping world areas into a precious commodity.
According to bankrate.com over 60 percent of millennial home buyer regret their home purchase. SHOCKING! In our industry this stat is overlooked and it has made me wonder why. I was working with someone who's lease was coming up and wanted to explore buying a home. They had two choices, rush into a home or wait another year. They decided to move and lease in the area they wanted to purchase. Being unfamiliar of the housing market after signing a 12-month lease there is no way to identify future homes that will come to the market when their lease is up. This story led us to a problem in our market. To avoid the traditional move-in ready market in less than 30 days, our users can match to homes that will be available for sale at their expected time to purchase. This is a perfect way to build confidence and prepare for the journey head.
Let's keep it 100 percent! We don't have access to who is planning for the future which could produce a better outcome. As a real estate professional over 10 years I’ve noticed soon after buyers move into their new home better homes in the neighborhood were listed for sale around the same price. We often times think we purchased the best home at the right time, but everyone has a plan and a price,That’s real estate right!
Would you be willing to wait if you found the home worth waiting for or made the seller an offer they can’t refuse for your one of a kind? Homematchx assures you never miss out on wishing you could have purchased the home next door, the home across the street, or the home around the corner. Our platform allow consumers to see available homes for sale up to three years out giving you access to more inventory than today.
I think seller are at a serious disadvantage when they need to sell. Days on Market is a huge issue and its a growing concern in real estate. They are unsure how many buyers fit their home's description and would be willing to purchase it at their desired time. To time selling your home perfectly is unpredictable. Our platform allows you to see all the buyers, their compatibility to your home, and if they have been qualified or not. Never will a seller list a home for sale without know who will actually purchase it.
We are heading into the new construction industry to help home builders better understand the real estate market and who's available. There is so many missing things that buyers don't have access to in order to time their new construction journey perfectly.
I'm excited about the many problems we can solve but I know we cannot be successful without the users knowing it exist. I'm on a Godly mission to finally change the real estate market and make it accessible regardless of your timeline.
Stephen L.
Biggest challenge has been speed: first proof of concept was a prototype that was kinda' slow (C#/WPF/Windows). I've re-written it using the lowest level possible stuf from WPF, and that took me a looot (roughly 3-4 months, to also make it easy to extend/modify). That was an improvement of roughtly 3-4 times, but for non-trivial stuff, it was too slow (and especially saving the final video was insaaaanely slow). So, I did another rewrite in UWP, and this took another 4+ months.
Now, I'm really happy about the speed - it's 3-4 times faster than before, and at saving, it's 10-12 times faster.
In order to make it happen, I've worked insane hours (and still am) - but that's that. Right now (the following 2 months) I'm focusing on stability and some improvements. Hope to have apretty cool new feature ready in roughly 3-4 months, and we'll see.
Challenges: countless, probably I could write a book ;)
1. Parsing existing videos - in WPF that was insanely hard, and it took me a lot of time to come up with a viable solution (which when porting to UWP, I ended up throwing away)
2. Estimates - I was pretty good at estimating how long a task would take. But due to the fact that everything was new to me (basically, animating using low level APIs was close to undocumented), so pretty much everything took 4-5 times more than I expected. This was soooo exhausting and depressing, since at some point I just stopped estimating, because I knew it would take me longer.
3. Changing the UI due to user feedback - basically, I ended up redesigning 80% of the UI to make it easier to use. What I thought would take me 1 week, ended up taking me 1+ months.
4. Tackleing everything at once: trying to implement a new feature, while dealing with bugs people would find or dealing with issues that would come up when trying to implement the feature. And dealing with issues that came up from the photographers I collaborate with (those that create the app's effects/transitions).
5. Porting to a new technology (UWP/WinRT). This is something that I hope I never have to do again - I was forced to do it, because of the speed gains. I had to reimplement / retest every control I initially developed - that's one thing. The other one is dealing with the idiocracy of WinRT - which loves async stuff / and also loves limitations. Also, the UWP documentation is soooo bad compared to WPF - and there are very few resources, because most people are put off by it (not going to go into detail as to why, that's another book I could write). Not for the faint of hearted. 6. Compilation times - on the old technology (WPF), everything was insanely awesome. On UWP, compilation times are roughly 6 times slower. That is baaaaaaaaad. I'm doing all sorts of workarounds to make things faster.
It's based on a university project on which I was working basically since day one in 2006.
I know it's crazy to work on such a large project initially alone. Lately, however, I'm getting the first contributions, and maybe I should start collaborating with the university or with the company of my former supervisor (who began the project for his Ph.D.).
I'm now more than convinced that the ideas are worth to work on, especially in the advent of modern hardware as byte-addressable NVM :-)
Currently, I'm working on the storage engine itself, to reduce storage space consumption further and to make the system stable. I'm experimenting with larger data sets to import (JSON and XML currently up to 5GB) with and without auto-commits, enabling/disabling different features, for instance, storing a rolling merkle hash for each node, storing the number of descendants, a path summary and so on.
Some of the features:
- the storage engine is written from scratch
- completely isolated read-only transactions and one read/write transaction concurrently with a single lock to guard the writer. Readers will never be blocked by the single read/write transaction and execute without any latches/locks.
- variable-sized pages
- lightweight buffer management with a "kind of" pointer swizzling
- dropping the need for a write-ahead log due to atomic switching of an UberPage
- rolling merkle hash tree of all nodes built during updates optionally
- ID-based diff-algorithm to determine differences between revisions taking the (secure) hashes optionally into account
- non-blocking REST-API, which also takes the hashes into account to throw an error if a subtree has been modified in the meantime concurrently during updates
- versioning through a huge persistent and durable, variable-sized page tree using copy-on-write
- storing delta page-fragments using a patented sliding snapshot algorithm
- using a special trie, which is especially good for storing records sith numerical dense, monotonically increasing 64 Bit integer IDs. We make heavy use of bit shifting to calculate the path to fetch a record
- time or modification counter-based auto-commit
- versioned, user-defined secondary index structures
- a versioned path summary
- indexing every revision, such that a timestamp is only stored once in a RevisionRootPage. The resources stored in SirixDB are based on a huge, persistent (functional) and durable tree
- sophisticated time travel queries
- How to shard databases
- Query compiler rewrite rules and cost-based optimization
- A brand new front-end
- Other secondary index-structures besides AVL trees stored in data nodes
- Storing graphs and other data types
- How to best make use of modern hardware as byte-addressable NVM
Anyway...
This summer & fall I wrote a JS core lib and a set of compatible packages that together greatly simplify the creation of terminal based node apps and games (in the realm of blessed, blessed-contrib and ink, but with no dependencies and with a novel api/architecture)
I got into it because my son did this node project where an animated car drove in a forest of cellular automata generated trees. Yah. You read it right. Things spiraled from there...
It is not a small project and it is pretty close to release form. I’ve used the lib and components to write a couple of small but non-trivial things. So, yes, it works.
In December, though, I stopped actively working on it. There are various reasons. One of which is that there is snow on the mountains. There are other reasons, none of which is code related.
More curious? More question. Cheers.
Quite a long story short, I managed to get the mental illness under control; something I thought I'd be living with the rest of my life.
My research has included mostly standing/walking meditation, and reading a lot on philosophy, religion, psychology, and such.
This is a personal project I've only just sort of revealed, after some persuasion by my peers. I didn't really have much intention on putting out in the public but it has turned out to be something significant. There is a lot to say about it.
EDIT: If you're curious, here is what I came up with after I started recording my research. DISCLAIMER: there is some personal stuff I talk about.
Thank you
I've always loved going on walks, and they've just turned out to be a good way to work through mental/emotional stuff. Walking helps you stay in the moment as well, and when you're focusing your attention on the moment there's always new things to sense.
Let me know if that's not clear enough.
EDIT:
Another important part of meditation is the analysis of your thoughts and emotions. Whatever comes up, try to understand why it did as well as you can.
About analyzing thoughts - I tend to over analyze everything. How do you not fall into this trap?
It's really a best effort sort of thing. But the more you practice, and the more effort you put forward, the stronger you'll get and the easier it will be to filter the signal from the noise.
Walking while meditating. There's a common misconception that you need to be sitting or lying down to meditate *or even need to be in a dark candle lite room lol). Meditating is a state of mind.
He's doing an overview of huge fields, not an in-depth, point-by-point argument. You sometimes have to fill in details of his argument yourself, because he doesn't answer every possible counter-argument.
Google query
"base of the natural logarithm e"
reports
e = 2.718281828459
that is, 13 digits.
The calculator with Windows 10 reports
e = 2.7182818284590452353602874713527
that is 32 decimal digits.
Last weekend found
e = 2.71828182845904523536028747135266250
that is, 36 decimal digits.
The math and code are below and could just as easily get e to, say, 500 decimal digits!
How'd that happen?
Last weekend worked on some short but relatively careful notes to get a nephew of 9 started on calculus, and part of that was Taylor series in just two pages with large fonts!
The code and the core of the Taylor series derivation are below.
In TeX, Taylor series is
f(x) = \sum_{i=0}^n {(x - x_0)^i \over i!} f^{[i]}(x_0) + R_n(x_0)
with R_n(x_0) as the error term.
To derive the Taylor series, really just find the error term
R_n(x_0)
and for that just differentiate f(x) with respect to x_0 where then nearly all the terms cancel, simplify, integrate from x_0 to x, and apply the mean value theorem. That's all there is to it!
The results are, for some s between x_0 and x:
R_n(x_0) = (x - x_0) {(x-s)^n \over n!} f^{[n+1]}(s)
As above, the final output of the code:
e = 2.71828182845904523536028747135266250
From R_n(x_0) the error is less than
3 x 10^(-40)
The numerical output of the code is curious: Get a little over 1 decimal digit of accuracy for each term of the series! So the output shows two big triangles, one for the values of n! and one for the number of correct digits in the estimate of e.
A key to why this code is so simple and works so well, Kexx can do arithmetic with 1000 decimal digits of precision!
"Look, Ma, here's the code -- dirt simple":
macro_name = 'NATLOG'
out_file = macro_name || '.out'
'nomsg erase' out_file
Call msgg macro_name': Find natual logarithm base e'
numeric digits 1000
n = 35
sum = 1
factorial = 1
Do i = 1 To n
factorial = i * factorial
sum = sum + 1/factorial
Call msgg Format(i, 5) Format(factorial, 50) Format(sum, 2, 35)
End
error = 3 / factorial
Call msgg macro_name': The error is <='
Call msgg Format( error, 59, 50 )
Call Lineout out_file
Return
msgg:
Procedure expose out_file
Call Lineout out_file, arg(1)
Return> notes to get a nephew of 9
I'm writing the notes for my 9 year old niece. So, the notes are for a boy of 9. And it's calculus, for a boy of 9. Of course it's "calculus 1".
I thought I mentioned it was for a boy of 9.
So, I omitted (1) a real valued function of a real variable with a compact domain has a Riemann integral if and only it is continuous on a set of measure zero, (2) for such function, if it is differentiable, then it has no jump discontinuities, (3) there is such a function that is differentiable but whose Riemann integral does not exist.
But, still, it is amazing how easy it is to get 36 decimal digits of e, and 500 if want.
I've been through calculus, advanced calculus, advanced calculus for applications, differential equations, local series solutions to the Navier-Stokes equations, exterior algebra, real analysis, measure theory, and more, have taught calculus in college, applied it in US national security and business, and published in it, and still I'd never seen a clear treatment of how easy it is to get so many digits of e from Taylor series.
The results I found are amazing, and my good and long experience indicates that only a tiny fraction of calculus students appreciate that.
And as we know,
e = lim_{i \rightarrow \infty} (1 + 1/i)^i
and it turns out that that iteration is painfully slow, and from my experience this fact is also amazing and poorly known.
It was amazing stuff and a productive weekend.