744 karma · joined September 20, 2013
[ my public key: https://keybase.io/vitorbaptista; my proof: https://keybase.io/vitorbaptista/sigs/b7rPmUpo5wAvOvwVoUdB0Hq_GEQWclN0mq4jwhL7Pxk ]
I also say that my concept of God is explained in a Simpsons intro [2]. It is based on the Powers of Ten video [3], where the camera is looking down increasing its height exponentially. First at 1 meter, then 10, 100, 1000, and so on. By the end, after we pass by the planets, solar systems, galaxies, and on, we end up in Homer's bald head. In other words, we're all part of the same entity; just like the bacteria in our guts is part of us, we're part of what we call God. And Homer (aka God) is not alone, which fits nicely in the many universes theory.
Many religions came to this conclusion, but I'm yet to find a better way to explain it than that Simpsons intro.
[1] https://waitbutwhy.com/2014/10/religion-for-the-nonreligious... [2] https://www.youtube.com/watch?v=FEuEx1jnt0M [3] https://www.youtube.com/watch?v=0fKBhvDjuy0
This december I decided to give YNAB [1] a try.
I was skeptical at first, but now I'm a convert. They have a lot of material on how to budget in a very simple way. But I think the moment their system clicked for me was when I noticed I was using Categories "wrong". Let me give an example.
Say my wife and I go out for dinner and movies. Previously, my expenses would look like:
Transportation - $20
Cinema - $40
Restaurant - $60
The categories are directly tied to what was purchased. That made sense, but this is focusing on the wrong level IMHO. It's not what was purchased, but why? With YNAB, my expenses look like:
Date Night - $120
The important isn't where the money was spent, but the reason you spent it. Paying a taxi to go to work is different than paying a taxi to go on a date.
This seemingly simple difference changed the way I view my budget. The categories are meaningful for me, and not just the standard "Groceries", "Transportation", "Rent", etc.
I have been using it just for one month, but I love it! I highly recommend you to give it a try.
Unless you're working with a lot of data, the technical parts of a data warehouse are relatively easy to grasp (the dbt team wrote some great posts on how they structure their projects [1]). After that, what really makes a difference is how you structure our data warehouse, how you model your data in a way that allow you to query it efficiently and ask many questions. That's where the Kimball books shine.
[1] https://community.snowflake.com/s/article/Use-Case-How-We-Co...
Luigi runs our scrapers and other workflow management tasks (e.g. DB backups).
All raw data lives in S3. We make an effort to be able to recreate the whole data warehouse from the raw data, so if any cleaning/normalization process fails, we have this safety net. I'm curious to hear if others use a similar pattern, or if there are better options.
DBT handles both loading the data from S3 into Snowflake (by creating the Snowflake Stages), and transforming the data in Snowflake. This isn't how DBT is used usually, but it felt wasteful to add Stitch or another tool to load the data into Snowflake, as snowflake supports it out of the box. I also created a `setup_snowflake` macro operation that creates our users, warehouses, databases, etc., in Snowflake (a kind of "poor man's Terraform")
I don't think Snowflake requires introduction. It's an amazing tool. We used Postgres before, but Snowflake is much much better, even though our DB is pretty small (~200 GB).
Finally, we use Re:dash as a BI, but I'm checking other options that allow usage without SQL (currently Metabase and Looker).
Do not vaccinate your children! A number of studies have shown that vaccines do NOT prevent disease. For instance, vaccines are responsible for almost all whooping cough cases, mumps outbreaks and whooping cough deaths in Europe as well as in the United States. Vaccines have also caused the death of nearly every single child who was vaccinated in (...)
One day I woke up suddenly after a bad dream. I was sitting on my bed alone. As soon as I turned over I found myself lying down in a strange bed. It had no bed, no mattress and no cover for me. I was just lying on that thick carpet. I looked around to discover nothing but a white bed. There was no window, no furniture. No door. The only clue was a single white mirror that stood beside my bed. And there was a single candle lying on my bedside table. I was not worried or scared. I was fascinated.
I could see the floor.
I could see the walls.
And I could see everything that was around me.
I could hear the wind.
I could see everything from the windows, from the door that was always slightly open. I could even see the light.
I was in a dream world and I knew that I was in a dream. But what I wanted to know was what was happening in this world.
I built a similar tool a while ago, but to share your entire (read-only) shell session interactively: https://shellshare.net/. Might be useful for some looking at this.
On most social networks (Facebook, Instagram, Reddit, etc.) the answer is simple: advertisement. To maximize revenue from ads, they focus on:
1) Getting people to spend as much time as possible on their platform (so they can show more ads), and; 2) Gathering as much information as possible about people (so their ads are more effective).
Which isn't exactly what people want when they go on Facebook. This puts the objectives of the social network and its users at odds.
Compare that to Hacker News, which is in itself an advertisement for Y Combinator (I wouldn't know of YC if it wasn't for HN, or at least it wouldn't be the first accelerator in my mind). I feel the objectives of Y Combinator with HN are much more aligned with my own objectives than Facebook's (the company) with Facebook (the social network).
I'd be worried when/if ads start appearing on HN...
Adding to this, the public sector pays much more than the private sector (often 50% ~ 100% more). Have job stability, generous vacations and other benefits. Depending on where you are (federal, state or municipal government levels), you also have a special retirement plan that gives you 100% of your salary when you retire (the private policies are capped at around 4.5 minimum wages, for comparison a good dev in the bigger cities can get about 10~15 minimum wages).
And so we have many "lifers" in the public sector, and this kind of job is the dream for many people (which is terrible IMHO).
Do you know if other people experienced the same?
I find it valuable to connect with friends, especially those I don't see often. If I know you've been traveling or got a new job, I can ask you about it. Instead of a generic "Hey, how are you doing?", I can go "How's the new job?". That makes it easier for me to have a small talk, so I connect more with friends, which is a good thing.
The problem is overdoing it. And everything is set for us to overdo it. It then stops being "cool, Jane seems to be having a great time, I'm happy for her!" to "everybody is having a blast while I'm stuck in 9-5, my life sucks".
The stories are basically turning people into vloggers. Depending on how often you post, you'll be in constantly thinking "should I be recording this?" or "what's the next shot?". This can be really draining for the person and their friends. There are few things that annoy me more than someone recording a video in a bar or a club. Even worse when they use flash.
Casey Neistat, of YouTube fame, is a great example of a "serial vlogger", someone that has been vlogging his life for decades. For example, on https://www.youtube.com/watch?v=kfIc944bSJ8 he talks about his girlfriend (now wife). He has footage from 2005 on their first date, hard conversations when they broke up, and (it seems) every highlight of their history. It makes an amazing video, I'd love to be able to rewatch those moments in my life. However, can you imagine living with a friend that's constantly recording their lives (including you)?
That's where we are headed...
Disclaimer: I work for the Open Knowledge International, which used to fund the Public Domain Review.
There's a brief explanation on why CSV was selected on https://specs.frictionlessdata.io/tabular-data-package/#why-...
We've scraped the Drugs@FDA data as part of OpenTrialsFDA [1], a project aimed to make it easier to search their documents. It contains data from about Oct/2016, because they changed their website's layout just after we finished the scraper (if you'd like to contribute, it shouldn't be difficult to update the scraper [2]). All the data is available in a Postgres database at https://explorer.opentrials.net/data, or via https://fda.opentrials.net.
I'm happy to talk with anyone interested in this kind of data via https://gitter.im/opentrials/chat.
[1]: https://opentrials.net/opentrialsfda/ [2]: https://github.com/opentrials/opentrials/issues/705
I don't see myself stopping anytime soon.
If you're interested in public spending data, take a look at OpenSpending (https://openspending.org/). It's a project that aims to make exploring, visualizing, and searching spending data easier. It's all free software (https://github.com/openspending) and you can upload your own datasets or explore others. It also provides you with an API, so you can use it to build finance-related projects (e.g. http://orcamento.inesc.org.br)
from:jhgg "Fallout 4" -Witcher
Is that so? If so, you created a parser for this format yourselves and are converting that to ElasticSearch's Query DSL?