hexdevs podcast

My wife and I are launching the greatest podcast in the world and a new site called hexdevs! We鈥檒l talk about how to start a business, and how to get better at your craft. We want everyone to be more confident, successful, and smarter, so if you want to become an awesome dev, listen to our podcast and sign up for the mailing list! And stay tuned! ...

July 17, 2019 路 1 min 路 Thiago Araujo

Generic JDBC Queries on EMR Zeppelin

The EMR (Elastic Map Reduce) service on Amazon has some nice packages that come pre-installed, and one of them is Apache Zeppelin, which is a Jupyter Notebook interface for Spark. Zeppelin has interpreters for spark, pyspark, spark-sql and others, but if you want to run spark-sql code on a PostgreSQL database, you need first to install the JDBC interpreter and add some extra configuration to Zeppelin. The JDBC adapter supports a wide variety of database engines, and it allows you to configure multiple database connections, which makes data exploration much easier....

April 3, 2019 路 5 min 路 Thiago Araujo

Fast ElasticSearch Indexing with Apache Spark on EMR (overview)

I鈥檝e been building the data infrastructure for a project and I needed to efficiently query, merge, process and clean terabytes of structured data and then index hundreds of millions of documents on elasticsearch. The problem is that querying and joining data on a RDBMS like Postgres is very painful when you have more than low terabytes of data. You鈥檙e going to spend a huge amount of time tuning your database, reading query plans, adding indexes, sharding, and slowly moving data around until you have something decent that take hours, days, maybe weeks to run....

September 6, 2018 路 4 min 路 Thiago Araujo

Machinations

An interesting quote about the unexpected effects of actions and the limitations of rationality and the planning fallacy: A net set up to catch fish may snare a duck; a mantis hunting an insect may itself be set upon by a sparrow. Machinations are hidden within machinations; changes arise beyond changes. So how can wit and cleverness be relied upon? 鈥 Back to Beginnings, Reflections on the Tao by Huanchu Daoren, translated by Thomas Cleary....

September 6, 2018 路 1 min 路 Thiago Araujo
xkcd: The Strong Collatz Conjecture states that this holds for any set of obsessively-hand-applied rules

Collatz Conjecture

The Collatz Conjecture is a simple mathematical problem that still has no formal proof. So it鈥檚 an open problem. This is how it works: Choose any positive integer x. While x > 0, do: if x is even, divide it by 2 (x = x/2) if x is odd, multiply it by 3 and add 1 (x = 3x + 1) The Collatz Conjecture states that no matter what value of x you start with, the sequence will always reach x = 1....

October 15, 2015 路 2 min 路 Thiago Araujo