race to get the world’s first plane in the sky was a hard fought battle
between The Wright Brothers and a lesser-known gentleman by the name of
Samuel Pierpont Langley.
You will discover why you’ve never heard of the latter here shortly.
you probably read somewhere inside that history textbook you were
forced to lug around through elementary — The Wright Brothers were
responsible for creating the first successful airplane. You remember how the story goes…
it was a cold windy day on December 17th, 1903 in the Kill Devil Hills
of North Carolina… Orville watched nervously as his brother Wilbur
climbed inside the plane they had spent years perfecting… miraculously
it flew for 59 seconds for a distance of 852 feet…”
While today “The Wright Brothers” is the first name that comes to anyone’s mind when they hear the word fly, once upon a time the pair were major underdogs.
In fact, during the race to the sky, most of America had its money on the man I mentioned earlier, Langley.
was an extremely outspoken astronomer, physicist and aviation pioneer
who was on a mission to make history. Langley’s high stature as the
Secretary of the Smithsonian Institution gave him both the credibility
and hype he needed to get America on his side.
to mention, he was extremely well-backed by the War Department who
contributed $50,000 to help him be the first to get a bird in the sky.
scientists more likely to write papers if they tell colleagues about
their intentions or if they keep their intentions to themselves?
Gollwitzer and his team of researchers carried out a handful of studies, here is a brief excerpt from their findings:
people’s taking notice of one’s identity-relevant intentions apparently
engenders a premature sense of completeness regarding the identity
In English, what Gollwitzer found was that when individuals set a goal that is closely tied to their identity and then share their intentions with others, they are less likely to achieve the goal.
example, if your goal is to start drinking more water and you tell your
friends and family that you’re going to start drinking more water, this
would probably have little to no impact on whether or not you actually
drink more water.
Why? Because drinking more water isn’t something you hold close to your identity.
the other hand, if your goal is to lose 40 lbs and drop 2–3 waist
sizes, it might not be the best idea to post about it all over Facebook.
Your appearance is something you very much so identify with. So, if you
tell people you plan to lose weight and everyone tells you how awesome
you are and how great you’re going to look, you might be less likely to
lose the weight.
finding is a bit counterintuitive, considering we were told by our
teachers and coaches growing up to set our goals, share our goals, hold
the theory certainly holds some weight (pun very much intended), and is
one that has been adopted by highly successful serial entrepreneurs
like Derek Sivers, founder of CD Baby.
Sivers gave a TED Talk
on this very topic nearly a decade back. To prove his point, he asked
the audience to imagine how they felt when they shared their goals with
their congratulations and their high image of you. Doesn’t it feel good
to say it out loud? Don’t you feel one step closer already? Like, it’s
already becoming part of your identity?
Well, bad news. You should have kept your mouth shut. That good feeling makes you less likely to do it.”
Sivers goes on to explain that it’s this “warm feeling” that keeps us from battling on to actually achieve our goals.
study put 800 undergraduate and graduate students at the University of
Pennsylvania through an 11-week exercise program where each person was
assigned to work out alone or in a team.
In addition, the teams were designed to be either supportive or competitive.
the end of the study, researchers found that students involved in the
competitive team programs were 90% more likely to attend their scheduled
exercise sessions than any other group.
only is this number staggering, but it also proves that competition can
create a higher level of commitment among people chasing down goals.
you surround yourself with competition, it doesn’t mean that you have
to share your goals with the competition. You don’t have to tell the
other folks in the spin class, cross-fit training or pick-up basketball
leagues that your goal is to lose 50 lbs.
by simply showing up and placing yourself in a competitive environment,
you will be more likely to push harder and show up more often — two
factors that can help your reach your goals.
The science behind achieving goals has always been an interesting topic.
post is making rounds on social media, in response to the Women’s March
on Saturday, January 21, 2017. It starts with “I am not a “disgrace to
women” because I don’t support the women’s march. I do not feel I am a
“second class citizen” because I am a woman….”
thank you. Say thank you to the women who gave you a voice. Say thank
you to the women who were arrested and imprisoned and beaten and gassed
for you to have a voice. Say thank you to the women who refused to back
down, to the women who fought tirelessly to give you a voice. Say thank
you to the women who put their lives on hold, who –lucky for you — did
not have “better things to do” than to march and protest and rally for
your voice. So you don’t feel like a “second class citizen.” So you get
to feel “equal.”
Thank Susan B. Anthony and Alice Paul for your right to vote.
Thank Elizabeth Stanton for your right to work.
Thank Maud Wood Park for your prenatal care and your identity outside of your husband.
Thank Rose Schneiderman for your humane working conditions.
Thank Eleanor Roosevelt and Molly Dewson for your ability to work in politics and affect policy.
Thank Margaret Sanger for your legal birth control.
Thank Carol Downer for your reproductive healthcare rights.
Thank Margaret Fuller for your equal education.
Ruth Bader Ginsburg, Shannon Turner, Gloria Steinem, Zelda Kingoff
Nordlinger, Rosa Parks, Angela Davis, Malika Saada Saar, Wagatwe
Wanjuki, Ida B. Wells, Malala Yousafzai. Thank your mother, your
grandmother, your great-grandmother who did not have half of the rights
you have now.
can make your own choices, speak and be heard, vote, work, control your
body, defend yourself, defend your family, because of the women who
marched. You did nothing to earn those rights. You were born into those
rights. You did nothing, but you reap the benefits of women, strong
women, women who fought misogyny and pushed through patriarchy and
fought for you. And you sit on your pedestal, a pedestal you are
fortunate enough to have, and type. A keyboard warrior. A fighter for
complacency. An acceptor of what you were given. A denier of facts.
Wrapped up in your delusion of equality.
are not equal. Even if you feel like you are. You still make less than a
man for doing the same work. You make less as a CEO, as an athlete, as
an actress, as a doctor. You make less in government, in the tech
industry, in healthcare.
still don’t have full rights over your own body. Men are still debating
over your uterus. Over your prenatal care. Over your choices.
You still have to pay taxes for your basic sanitary needs.
still have to carry mace when walking alone at night. You still have to
prove to the court why you were drunk on the night you were raped. You
still have to justify your behavior when a man forces himself on you.
still don’t have paid (or even unpaid) maternity leave. You still have
to go back to work while your body is broken. While you silently suffer
from postpartum depression.
still have to fight to breastfeed in public. You still have to prove to
other women it’s your right to do so. You still offend others with your
are still objectified. You are still catcalled. You are still
sexualized. You are still told you’re too skinny or you’re too fat.
You’re still told you’re too old or too young. You’re applauded when you
“age gracefully.” You’re still told men age “better.” You’re still told
to dress like a lady. You are still judged on your outfit instead of
what’s in your head. What brand bag you have still matters more than
your college degree.
are still being abused by your husband, by your boyfriend. You’re still
being murdered by your partners. Being beaten by your soulmate.
are still worse off if you are a woman of color, a gay woman, a
transgender woman. You are still harassed, belittled, dehumanized.
daughters are still told they are beautiful before they are told they
are smart. Your daughters are still told to behave even though “boys
will be boys.” Your daughters are still told boys pull hair or pinch
them because they like them.
You are not equal. Your daughters are not equal. You are still systematically oppressed.
Estonia allows parents to take up to three years of leave, fully paid for the first 435 days. United States has no policy requiring maternity leave.
Singapore’s women feel safe walking alone at night. American women do not.
New Zealand’s women have the smallest gender gap in wages, at 5.6%. United States’ pay gap is 20%.
Iceland has the highest number of women CEOs, at 44%. United States is at 4.0%.
I get it. You don’t want to admit it. You don’t want to be a victim.
You think feminism is a dirty word. You think it’s not classy to fight
for equality. You hate the word pussy. Unless of course you use it to
call a man who isn’t up to your standard of manhood. You know the type
of man that “allows” “his” woman to do whatever she damn well pleases. I
get it. You believe feminists are emotional, irrational, unreasonable.
Why aren’t women just satisfied with their lives, right? You get what
you get and you don’t get upset, right?
get it. You want to feel empowered. You don’t want to believe you’re
oppressed. Because that would mean you are indeed a “second-class
citizen.” You don’t want to feel like one. I get it. But don’t worry. I
will walk for you. I will walk for your daughter. And your daughter’s
daughter. And maybe you will still believe the world did not change. You
will believe you’ve always had the rights you have today. And that’s
okay. Because women who actually care and support other women don’t care
what you think about them. They care about their future and the future
of the women who come after them.
your eyes. Open them wide. Because I’m here to tell you, along with
millions of other women that you are not equal. Our equality is an
illusion. A feel-good sleight of hand. A trick of the mind. I’m sorry to
tell you, but you are not equal. And neither are your daughters.
don’t worry. We will walk for you. We will fight for you. We will stand
up for you. And one day you will actually be equal, instead of just
feeling like you are.
Altcoins are dying a coin and code death faster than ever before.
cryptocurrency market is sadly tethered to Bitcoin’s price. Bitcoin’s
price is probably at the mercy of large-scale market manipulation and
cryptic factors that even the crypto frauds and “decentralization”
pretenders can’t simulate.
the RIP Crypto title here is still just a meme. There’s no end for
crypto in sight, with the levels of blockchain projects we are seeing,
with the levels of daily active developers being attracted to the new
space and bigger investors, crypto mania is only a faltering global
economy away. Most economics peg the next global recession around twelve
to eighteen months away, so late 2019 or early 2020.
RIP “crypto”. You had a good run.
Wall Street and Bloomberg reports tempted to take this literally, and
try to convince you that crypto really is dead, there was one funny
thing that did occur recently.
week veteran cryptographer Matt Blaze, according to TechCrunch, did
finally sell the pithy domain name he registered in 1993, in the midst
of the PC era crypto wars — to none other than Crypto Valley.
The buyer/recipient is Monaco,
the Zug, Switzerland-based payments and cryptocurrency platform startup
whose self-styled mission is “accelerating the world’s transition to
cryptocurrency”, positioning itself at the nexus of the current crypto
got a lot of sneaky startups Crypto! You are vilified but sort of
immortal in our psyches. Everyone likes a dark horse though, as young
consumers are themselves penalized in a global economy stacked against
them. Crypto, is like a symbol of an uprising. An uprising in coin, code, node and word-of-mouth Bitcoin gibberish.
It’s that recognition on a global level, that decentralization really is the ethical corporate future of all services, government and digital infrastructure on the blockchain.
Listening to Vitalik Buterin
is like listening to the voices of the spirit of crypto, itself. You
want to support what he does, but you don’t know wtf he is talking
about. I suspect EOS will fail and NEO will vastly improve its
offering before 2020. But what do I know, I’m just a beginner blockchain
historian of sorts. Ethereum’s sharding, plasma and Casper could make
contenders like Waves, Tezos, Stratis, Qtum and others obsolete.
you think Crypto is dead, just go on Reddit and check out all of the
subreddits around them. They are some of the most active and fun
subreddits to follow on all of Reddit, that itself has massively engaged
users. Slack and Telegram users migrating to Discord, is another trend
that shows blockchain startups are building amazing communities. Even as
Facebook wants to enable Facebook Groups to be monetized like exclusive
clubs. This from a platform that has banned Crypto Ads, and
perma-banned ICO Ads.
very difficult to see Bitcoin dying any time soon, blockchain patents
are being activated now. The Ethereum Foundation tries very hard to be a
decentralization community; basically a new kind of model from the
Microsoft Cloud developers model (that recently bought out GitHub).
Developers want to be free and contribute their talents for some kind of
a planetary good. Ethereum’s ecosystem is a place for collaboration;
not some kind of centralized model like Microsoft; or even the likes of
Ripple, Block.One and other experiments.
DFINITY, VeChain, TRON and so many other dApp and MainNets deserve
close watching. Blockchain startups aren’t even going at full speed yet,
altcoins could explode if another crypto singularity took place (sort
of like the bump of December, 2017). Over 99% of the use cases of
blockchain have yet to even mature. RIP Crypto, yes O’ really?
future is usability and scalability, privacy and security are on-going
challenges. The majority of the most valuable cryptocurrencies are dApp,
smart contract, platforms and developer ecosystems that could scale
with solid real-world capabilities for a future we cannot even quite
sale of crypto.com domain for like $10 million doesn’t help us
understand the price volatility or regulatory immaturity of the space;
all it tells us is that crypto is a beast that is two-faced. When the
future of coin and code is at stake, there’s bound to be a lot of
cannot die because it’s also short for cryptography; and we all know
the future of cybersecurity is one of the major issues of the evolution
of the web, the internet of things and all things mobile, markets, data,
privacy, and fraud. How anonymity meets user-ID verification meets
taxes, meets dark-net transactions and side hustle investments, is bound
to have some growing pains. If anything points to crypto not dying,
it’s Coinbase custody.
long as man has been alive, he has been collecting little sayings about
how to live. We find them carved in the rock of the Temple of Apollo
and etched as graffiti on the walls of Pompeii. They appear in the plays
of Shakespeare, the commonplace book
of H. P. Lovecraft, the collected proverbs of Erasmus, and the ceiling
beams of Montaigne’s study. Today, they’re recorded on iPhones and in
whatever generation is doing it, whether they’re written by scribes in
China or commoners in some European dungeon or simply passed along by a
kindly grandfather, these little epigrams of life advice have taught
essential lessons. How to respond to adversity. How to think about
money. How to meditate on our mortality. How to have courage.
they pack all this in in so few words. “What is an epigram?” Coleridge
asked, “A dwarfish whole; Its body brevity, and wit its soul.” Epigrams
are what Churchill was doing when he said: “To improve is to change, so
to be perfect is to have changed often.” Or Balzac: “All happiness
depends on courage and work.” Ah yes, epigrams are often funny too.
That’s how we remember them. Napoleon: “Never interrupt an enemy making a
mistake.” François de La Rochefoucauld: “We hardly find any persons of
good sense save those who agree with us.” Voltaire: “A long dispute
means that both parties are wrong.”
are some wonderful epigrams that span some 21 centuries and 3
continents. Each one is worth remembering, having queued in your brain
for one of life’s crossroads or to drop at the perfect moment in
conversation. Each will change and evolve with you as you evolve
(Heraclitus: “No man steps in the same river twice”) and yet each will
remain strong and unyielding no matter how much you may one day try to
wiggle out and away from them.
Fundamentally, each one will teach you how to be a better person. If you let them.
“We must all either wear out or rust out, every one of us. My choice is to wear out.” — Theodore Roosevelt
At the beginning of his life, few would have predicted that Theodore Roosevelt
even had a choice in the matter. He was sickly and fragile, doted on by
worried parents. Then, a conversation with his father sent him driven,
almost maniacally in the other direction. “I will make my body,”
he said, when told that he would not go far in this world with a
brilliant mind in a frail body. What followed was a montage of boxing,
hiking, horseback riding, hunting, fishing, swimming, boldly charging
enemy fire, and then a grueling work pace as one of the most prolific
and admired presidents in American history. Again, this epigram was
prophetic for Roosevelt, because at only 54 years old, his body began to
wear out. An assassination attempt left a bullet lodged in his body and it hastened his rheumatoid arthritis. On his famous “River of Doubt”
expedition he developed a tropical fever and the toxins from an
infection in his leg left him nearly dead. Back in America he contracted
a severe throat infection and was later diagnosed with inflammatory
rheumatism, which temporarily confined him to a wheelchair (saying
famously, “All right! I can work that way too!”) and then he died at age
60. But there is not a person on the planet who would say that he had
not made a fair trade, that he had not worn his life well and not lived a
full one in those 60 years.
“It’s not what happens to you, but how you react to it that matters.” — Epictetus
is the story of the alcoholic father with two sons. One follows in his
father’s footsteps and ends up struggling through life as a drunk, and
the other becomes a successful, sober businessman. Each are asked: “Why
are you the way you are?” The answer for both is the same: “Well, it’s
because my father was an alcoholic.” The same event, the same childhood,
two different outcomes. This is true for almost all situations — what
happens to us is an objective reality, how we respond is a subjective
choice. The Stoics — of which Epictetus was one
— would say that we don’t control what happens to us, all we control
are our thoughts and reactions to what happens to us. Remember that:
You’re defined in this life not by your good luck or your bad luck, but
your reaction to those strokes of fortune. Don’t let anyone tell you
“The best revenge is not to be like that.” — Marcus Aurelius
There is a proverb about revenge: Before setting out for a journey of revenge, dig two graves.
Because revenge is so costly, because the pursuit of it often wears on
the one who covets it. Marcus’s advice is easier and truer: How much
better it feels to let it go, to leave the wrongdoer to their
wrongdoing. And from what we know, Marcus Aurelius
lived this advice. When Avidius Cassius, one of his most trusted
generals rebelled and declared himself emperor, Marcus did not seek
vengeance. Instead, he saw this as an opportunity to teach the Roman
people and the Roman Senate about how to deal with civil strife in a
compassionate, forgiving way. Indeed, when assassins struck Cassius
down, Marcus supposedly wept. This is very different than the idea of
“Living well being the best revenge” — it’s not about showing someone up
or rubbing your success in their face. It’s that the person who wronged
you is not happy, is not enjoying their life. Do not become like them.
Reward yourself by being the opposite of them.
“There is good in everything, if only we look for it.” — Laura Ingalls Wilder
Laura Ingalls Wilder, author of the classic series Little House,
lived this, facing some of the toughest and unwelcoming elements on the
planet: harsh and unyielding soil, Indian territory, Kansas prairies,
and the humid backwoods of Florida. Not afraid, not jaded — because she
saw it all as an adventure. Everywhere was a chance to do something new,
to persevere with cheery pioneer spirit whatever fate befell her and
her husband. That isn’t to say she saw the world through delusional
rose-colored glasses. Instead, she simply chose to see each situation
for what it could be — accompanied by hard work and a little upbeat
spirit. Others make the opposite choice. Remember: There is no good or
bad without us, there is only perception. There is the event itself and
the story we tell ourselves about what it means.
the hiring process, most employers look at where someone went to
school, what jobs they’ve held in the past. This is because past success
can be an indicator of future successes. But is it always? There are
plenty of people who were successful because of luck. Maybe they got
into Oxford or Harvard because of their parents. And what about a young
person who hasn’t had time to build a track record? Are they worthless?
Of course not. This is why character is a far better
measure of a man or woman. Not just for jobs, but for friendships,
relationships, for everything. When you seek to advance your own
position in life, character is the best lever — perhaps not in the short
term, but certainly over the long term. And the same goes for the
people you invite into your life.
“If you see fraud and do not say fraud, you are a fraud.” — Nicholas Nassim Taleb
man shows up for work at a company where he knows that management is
doing something wrong, something unethical. How does he respond? Can he
cash his checks in good conscience because he isn’t the one running up
the stock price, falsifying reports or lying to his co-workers? No. One
cannot, as Budd Schulberg says in one of his novels,
deal in filth without becoming the thing he touches. We should look up
to a young man at Theranos as an example here. After discovering
numerous problems at the health care startup, he was dismissed by his
seniors and eventually contacted the authorities. Afterwards, not only
was this young man repeatedly threatened, bullied, and attacked by
Theranos, but his family had to consider selling their house to pay for
the legal bills. His relationship with his grandfather — who sits on the
Theranos board — is strained and perhaps irreparable. As Marcus Aurelius reminded himself,
and us: “Just that you do the right thing. The rest doesn’t matter.”
It’s an important reminder. Doing the right thing isn’t free. Doing the
right thing might even cost you everything.
“Every man I meet is my master in some point, and in that I learn of him.” — Ralph Waldo Emerson
is better than you at something. This is a fact of life. Someone is
better than you at making eye contact. Someone is better than you at
quantum physics. Someone is better informed than you on geopolitics.
Someone is better than you are at speaking kindly to someone they
dislike. There are better gift-givers, name-rememberers, weight-lifters,
temper-controllers, confidence-carriers, and friendship-makers. There
is no one person who is the best at all these
things, who doesn’t have room to improve in one or more of them. So if
you can find the humility to accept this about yourself, what you will
realize is that the world is one giant classroom. Go about your day with
an openness and a joy about this fact. Look at every interaction as an
opportunity to learn from and of the people you meet. You will be amazed
at how quickly you grow, how much better you get.
“This is not your responsibility but it is your problem.” — Cheryl Strayed
is not your responsibility to fill up a stranger’s gas tank, but when
their car dies in front of you, blocking the road, it’s still your
problem isn’t it? It is not your responsibility to negotiate peace
treaties on behalf of your country, but when war breaks out and you’re
drafted to fight in it? Guess whose problem it is? Yours.
Life is like this. It has a way of dropping things into our lap — the
consequences of an employee’s negligence, a spouse’s momentary lapse of
judgement, a freak weather event — that were in no way our fault but by
nature of being in our lap, our f*cking problem. So what are you going to do? Complain?
Are you going to litigate this in a blogpost or an argument with God?
Or are you just going to get to work solving it the best you can? Life
is defined by how you answer that question. Cheryl Strayed is right. This thing might not be your responsibility but it is your problem. So accept it, deal with it, kick its ass.
“Waste no more time arguing what a good man should be. Be one.” — Marcus Aurelius
Rome just as America, in the forum just as on Facebook, there was the
temptation to replace action with argument. To philosophize instead of living philosophically. Today, in a society obsessed with content, outrage,
and drama, it’s even easier to get lost in the echo chamber of the
debate of what’s “better.” We can have endless discussions about what’s
right and wrong. What should we do in this hypothetical situation or
that one? How can we encourage other people to be better? (We can even
debate the meaning of the above line: “What’s a man? What’s the
definition of good? Why doesn’t it mention women?”) Of course, this is
all a distraction. If you want to try to make the world a slightly
better place, there’s a lot you can do. But only one thing guarantees an
impact. Step away from the argument. Dig yourself out of the rubble.
Stop wasting time with how things should be, would be, could be. Be that thing. (Here’s a cool poster of this quote).
“You are only entitled to the action, never to its fruits.” — Bhagavad Gita
life, it’s a fact that: You will be unappreciated. You will be
sabotaged. You will experience surprising failures. Your expectations
will not be met. You will lose. You will fail. How do you carry on then?
How do you take pride in yourself and your work? John Wooden’s advice
to his players says it: Change the definition of success.
“Success is peace of mind, which is a direct result of self
satisfaction in knowing you made the effort to do your best to become
the best that you are capable of becoming.” “Ambition,” Marcus Aurelius reminded himself,
“means tying your well-being to what other people say or do . . .
Sanity means tying it to your own actions.” Do your work. Do it well.
Then “let go and let God.” That’s all there needs to be. Recognition and
rewards — those are just extra.
“Self-sufficiency is the greatest of all wealth.” — Epicurus
lot has been said of so-called “F*ck You Money.” The idea being that if
one can earn enough, become rich and powerful enough, that suddenly no
one can touch them and they can do whatever they want. What a mirage
this is! How often the target seems to mysteriously move right as we
approach it. It calls to mind the observation of David “DHH” Heinemeier
Hansson who said that
“beyond a specific amount, f*ck-you money can be a state of mind. One
that you can acquire well in advance of the corresponding bank account.
One that’s founded mostly on a personal confidence that even if most of
the material trappings went away, you’d still be happier for standing
your ground.” The truth is being your own man, being self-contained,
having fewer needs, and better, resilient skills that allow you to
thrive in any and all situations. That is real wealth and freedom.
That’s what Emerson was talking about in his famous essay on self-reliance and it’s what Epicurus meant too.
“Tell me to what you pay attention and I will tell you who you are.” — Jose Ortega y Gasset
It was one of the great Stoics
who said that if you live with a lame man, soon enough you will walk
with a limp. My father told me something similar as a kid: “You become
like your friends.” It is true not just with social influences but
informational ones too: If you are addicted to the chatter of the news,
you will soon find yourself worried, resentful, and perpetually
outraged. If you consume nothing but escapist entertainment, you will
find the real world around you harder and harder to deal with. If all
you do is watch the markets and obsess over every fluctuation, your
worldview will become defined by money and gains and losses. But if you
drink from deep, philosophical wisdom? If you have regularly in your
mind role models of restraint, sobriety, courage, and honor? Well, you
will start to become these things too. Tell me who you spend time with, Goethe said,
and I will tell you who you are. Tell me what you pay attention to,
Gasset was saying, and I can tell you the same thing. Remember that the
next time you feel your finger itching to pull up your Facebook feed.
“Better to trip with the feet than with the tongue.” — Zeno
You can always get up after you fall, but remember, what has been said can never be unsaid. Especially cruel and hurtful things.
“Space I can recover. Time, never.” — Napoleon Bonaparte
can be reconquered, indeed in the course of a battle, a hill or a
certain plain might trade hands several times. But missed opportunities?
These can never be regained. Moments in time, in culture? They can
never be re-made. One can never go back in time to prepare for what they
should have prepared for, no one can ever get back critical seconds
that were wasted out of fear or ego.
Napoleon was brilliant at trading space for time: Sure, you can make
these moves, provided you are giving me the time I need to drill my
troops, or move them to where I want them to be. Yet in life, most of us
are terrible at this. We trade an hour of our life here or afternoon
there like it can be bought back with the few dollars we were paid for
it. And it is only much much later, as they are on their deathbeds or
when they are looking back on what might have been, that many people
realize the awful truth of this quote. Don’t do that. Embrace it now.
“You never know who’s swimming naked until the tide goes out.” — Warren Buffett
problem with comparing yourself to other people is you really never
know anyone else’s situation. The co-worker with a nice car? It could be
a dangerous and unsafe salvage with 100,000 miles. The friend who
always seems to be traveling to far off places? They could be up to
their eyeballs in credit card debt and about to get fired by their boss.
Your neighbors’ marriage which makes you so insecure about your own? It
could be a nightmare, a complete lie. People do a very good job
pretending at things, and their well-maintained fronts are often covers
for incredible risk and irresponsibility. You never know, Warren Buffett
was saying, until things get bad. If you’re living the life you know to
be right, if you are making good, solid decisions, don’t be swayed by
what others are doing — whether that is taking the form of irrational
exuberance or panicked pessimism. See the high flying lives of others as
a cautionary tale — like Icarus with his wings — and not as an
inspiration or a source of insecurity. Keep doing what you’re doing and
don’t be caught swimming naked! Because the tide will go out. Prepare
for it! (Premeditatio Malorum)
“Search others for their virtues, thyself for thy vices.” — Benjamin Franklin
would say something similar: “Be tolerant with others and strict with
yourself.” Why? For starters because the only person you control is
yourself. It’s a complete waste of time to go around projecting strict
standards on other people — ones they never agreed to follow in the
first place — and then being aghast or feel wronged when they fall
short. The other reason is you have no idea what other people are going
or have been through. That person who seemed to rudely decline the
invitation you so kindly offered? What if they were working hard to
recommit themselves to their family and as much as they’d like to have
coffee with you, are doing their best to spend more time with their
loved ones? The point is: You have no idea. So give people the benefit
of the doubt. Look for good in them, assume good in them, and let that
good inspire your own actions.
“The world was not big enough for Alexander the Great, but a coffin was.” — Juvenal
the way that a good one liner can humble even the world’s greatest
conqueror. Remember: we are all equals in death. It makes quick work of
all of us, big and small. I carry a coin in my pocket to remember this: Memento Mori. What Juvenal reminds us is the same thing that Shakespeare spoke about in Hamlet:
“Imperious Caesar, dead and turned to clay, Might stop a hole to keep the wind away. O’ that that earth which kept the world in awe Should patch a wall t’ expel the winder’s flaw!”
doesn’t matter how famous you are, how powerful you are, how much you
think you have left to do on this planet, the same thing happens to all
of us, and it can happen when we least expect it. And then we will be
wormfood and that’s the end of it.
“To improve is to change, so to be perfect is to have changed often.” — Winston Churchill
While this is probably not a Churchill original (he most likely borrowed
from Cardinal Newman: “In a higher world it is otherwise, but here
below to live is to change, and to be perfect is to have changed
often”), Churchill certainly abided this in his life.
He’d even quip about his constant change of political affiliation: “I
said a lot of stupid things when I worked with the Conservative Party,
and I left it because I did not want to go on saying stupid things.” As
Cicero would say when attacked that he was changing his opinion: “If
something strikes me as probable, I say it; and that is how, unlike
everyone else, I remain a free agent.” There is
nothing more impressive — intellectually or otherwise — than to change
long held beliefs, opinions, and habits. The more you’ve changed, the
better you probably are.
“Judge not, lest you be judged.” — Jesus
only here would Jesus call us on one of our worst tendencies but
immediately also ask: “And why do you look at the speck in your
brother’s eye, but do not consider the plank in your own eye?” This line
is similar to what the Stoic philosopher Seneca, who historical sources suggest was born the same year as Jesus,
would say: “You look at the pimples of others when you yourselves are
covered with a mass of sores.” Waste no time judging and worrying about
other people. You have plenty of problems to deal with in your own life.
Chances are your own flaws are probably worse — and in any case, they
are at least in your control. So do something about them.
“Time and patience are the strongest warriors.” — Leo Tolstoy
Tolstoy puts the above words in the mouth of Field Marshall Mikhail Kutuzov in War and Peace.
In real life, Kutuzov gave Napoleon a painful lesson in the truth of
the epigram over a long winter in Russia in 1812. Tolstoy would also
say, “Everything comes in time to him who knows how to wait.” When it
comes to accomplishing anything significant, you are required to exhibit
patience and fortitude, so much patience, as much as you’d think you’d need boldness and courage. In my book Conspiracy, about Peter Thiel’s plot to destroy Gawker, his operative describes a similar idea: With enough time and patience, you can do anything.
“No one saves us but ourselves / No one can and no one may.” — Buddha
we wait for someone to save us, or will we listen to Marcus Aurelius’s
empowering call to “get active in your own rescue — if you care for
yourself at all — and do it while you can.”
at some point, we must put articles like this one aside and take
action. No one can blow our nose for us. Another blog post isn’t the
answer. The right choices and decisions are. Who knows how much time you
have left, or what awaits us tomorrow? So get to it.
Like to Read?
I’ve created a list of 15 books you’ve never heard of that will alter your worldview and help you excel at your career.
me, the answer is obvious: by turning to wisdom. Below is a list of 21
books that will help lead you to a better, stronger 2018.
Deep Work: Rules for Focused Success in a Distracted Worldby Cal Newport Media consumption went way up in 2017. For most of us,
that meant happiness and productivity went way down. The world is
becoming noisier and will become more so every day. If you can’t
cultivate the ability to have quiet, insightful, deeply focused periods
of productive work, you’re going to get screwed. This is a book that
explains how to cultivate and protect that skill — the ability to do deep work. I strongly urge you to begin this practice in 2018— if you want to get anything done or perform your best.
The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Lifeby Mark Manson To me, practical philosophy has always been the art knowing what to — and what not to — give a fuck about.
That’s what Mark’s book is about. It’s not about apathy. It’s about
cultivating indifference to things that don’t matter. Be careful, as Marcus Aurelius warns,
not to give the little things more time and thought they deserved.
Maybe looking back at this year reveals how much effort you’ve frittered
away worrying about the trivial. If so, let 2018 be a year that you
only devote energy to things that truly matter — get the important
things right by ignoring the insignificant.
The Way to Love: The Last Meditations of Anthony de Mello by Anthony de Mello Coach
Shaka Smart recommended this little book (and it’s a little book,
probably the smallest I’ve ever read. It fits in your palm). But it’s an
incredibly wise and helpful read. Written by a Catholic Priest who’d
lived in India, the book has this unusual convergence of eastern and
western thought. One of my favorite lines: “The question to ask is not
‘What’s wrong with this person?’ but ‘What does this irritation tell me
about myself?’ I plan on regularly revisiting it throughout 2018.
But What If We’re Wrongby Chuck Klosterman It’s
always good to remind ourselves that almost everything we’re certain
about will probably be eventually proven wrong. Klosterman’s
subtitle — Thinking About the Present As If It Were the Past — is a
brilliant exercise for getting some perspective in 2018. Whether you
think it’s going to be a year of radical change for the better or a
horrible year of excesses of dangerous precedent, you’re probably wrong.
You’re probably not even in the ballpark. This book shows you why, not
with lectures about politics, but with a bunch of awesome thought
experiments about music, books, movies and science.
Rules for Radicals: A Practical Primer for Realistic Radicalsby Saul Alinsky If
Hillary Clinton had remembered the lessons of Saul Alinsky (who she
wrote her college thesis on), the election may have turned out
differently. Why? A notorious strategist and community organizer,
Alinsky was a die hard pragmatist, but he also knew how to tell a story
and create a collective cause. He could work within the system but knew
how to shake it up and generate attention. This book is a classic and
woefully underrated. Whatever you set out to do in 2018, this book can
provide you with strategic guidance and insight.
The Filter Bubble by Eli Pariser / Trust Me I’m Lying by Ryan Holiday / The Brass Check by Upton Sinclair I
strongly recommend that you take the time in 2018 to read these books.
In light of this year, you owe it to yourself to study and better
understand how our media system works. In The Filter Bubble,
Eli Pariser warns of the danger of living in bubbles of personalization
that reinforce and insulate our worldview. Though Sinclair’s The Brass Check
has been almost entirely forgotten by history, it’s not only
fascinating but a timeless perspective. Sinclair deeply understood the
economic incentives of early 20th century journalism and thus could
predict and analyze the manipulative effect it had on The Truth. I used
that book as a model for my expose of the media system, Trust Me, I’m Lying.
Today, the incentives and pressures are different but they warp our
information in a similar way. In almost every substantial charge Upton
leveled against the yellow press, you could, today, sub in blogs and the
cable news cycle and be even more correct.
48 Laws of Power / 33 Strategies of Warby Robert Greene Robert
Greene is a master of human psychology and human dynamics — he has a
profound ability to explain timeless truths through story and example.
You can read the classics and not always understand the lessons. But if
you read Robert’s books, I promise you will leave not just with
actionable lessons but an indelible sense of what to do in many trying
and confusing situations. I wrote earlier this year
that strategic wisdom is not something we are born with — but the
lessons are there for us to pick up. Pick these two up before the year
ends and operate the next with a strategic mindset and clarity.
Conspiracy: Peter Thiel, Hulk Hogan, Gawker, and the Anatomy of Intrigue by Ryan Holiday — If
you want to immerse yourself in the above topics of media and strategy,
and are looking for one book to teach you lessons in both, my book on
the nearly decade-long conspiracy that billionaire Peter Thiel waged
against Gawker will do this for you. This is a stunning story about how
power works in the modern age, and is a masterclass in strategy and how
to accomplish wildly ambitious aims.
The Road To Characterby David Brooks When General Stanley McChrystal was asked on the Tim Ferriss podcastwhat
was a recent purchase that had most positively impacted his life, he
pointed to this book. I agree. It can be a bit stilted and dense at
times, but it should be assigned reading to any young person today (a
little challenge is a good thing). Illustrating with examples and
stories from great men and women, Brooks admonishes the reader to
undertake their own journey of character perfection. In my own book, I explore the same topic (humility) from a different angle using similar stories — I’m attacking ego, he’s building up character. Both will be important for the next year.
The Dipby Seth Godin This
book is a short 70 pages and it looks like something someone would give
as a joke gift, but it’s anything but. Godin talks frankly about
quitting and pushing through — and when to do each. Quit
when you’ll be mediocre, when the returns aren’t worth the investment,
when you no longer think you’ll enjoy the ends. Stick when the dip is
the obstacle that creates scarcity, when you’re simply bridging the gap
between beginner’s luck and mastery. I promise, next year you are
guaranteed to find yourself in moments when you don’t know what is the
right answer. This book will help you find it.
Hillbilly Elegy: A Memoir of a Family and Culture in Crisis by J. D. Vance / Strangers in Their Own Land: Anger and Mourning on the American Rightby Arlie Russell Hochschild You might describe Hillbilly Elegy as a Ta-Nehisi Coates style memoir
about a community that — at least in progressive circles — gets a lot
less attention: disaffected, impoverished whites (particularly in the
mid-east and South). I thought the book was empathetic, self-aware and
inspiring. The author pokes some holes in the concept of “white
privilege” — certainly a third or fourth generation hillbilly in
Kentucky doesn’t walk around feeling like they have it easy — and an
explanation of some of the phenomenon behind Donald Trump
(notice I said explanation, not an excuse). This is a sober but also
hopeful book. I urge people on all sides of the political spectrum to
read it, but progressives would benefit from the eye-opening perspective
the most. Pair it with Strangers in Their Own Land,
the author of which, a Berkeley native, visits and interviews people in
the deep Louisiana countryside. (Remember: you don’t have to agree with
the people you’re reading about — but you should care what makes them
It Can’t Happen Hereby Sinclair Lewis This
book is one that will make you so uncomfortable that you’ll probably
pick it up and put it down several times. It almost shocks you that this
exists, that it’s not some work of fiction pretending to be 80 years
old. But no, in fact, one of America’s most famous writers wrote a
bestselling book about an appalling populist demagogue who won the
presidency of the United States. Europe was a mess, the economy was in
the toilet. Well meaning people talked earnestly about the need for
“serious change” and “revolution.” The parties split, and a fringe
candidate suddenly becomes viable. When people tried to question some of
the ideas he campaigned on, they were shouted down: “This is how all
politicians speak. He’s not serious about that extreme stuff.” Life
imitates art and now, this is what actually happened.
Change the dates, places and names and it’s no longer fiction, it’s
real. Fiction is best when it puts a mirror up to us. This book does
that. If you don’t read the book, at least please read about it. Because you need to know. It can happen here.
How to Live: Or A Life of Montaigne in One Question and Twenty Attempts at an Answer by Sarah Bakewell The book is spectacular. It was a bestseller in the UK and was featured in a 6 part series in The Guardian.
The format of the book is a bit unusual, instead of chapters it is made
up of 20 Montaigne style essays that discuss the man from a variety of
different perspectives. Montaigne was a man obsessed with figuring
himself out — why he thought the way he did, how he could find
happiness, his fetishes, his near-death experiences. He lived in
tumultuous times too and he coped by looking inward. We’re lucky he did,
and we can do the same.
Tribe: On Homecoming and Belonging by Sebastian Junger Junger
looks at many of the social issues we have today: PTSD. Depression.
Political polarization. Contempt for our fellow citizens. Aimlessness.
We’ve lost the bonds that tie a tribe and culture together. No wonder
veterans feel alone and lost when they leave their tight knit tribe and
re-enter solitary civilian life. No wonder manypeople
feel that way regardless of whether they served or not. No wonder we
have trouble agreeing on basic solutions to common problems. There is no sense of what is common or basic or shared.
Given the divisiveness that we are facing as a society — that became
painfully clear in 2016 — this is one of the most urgent and important
book you need to read next year.
The Years of Lyndon Johnson (4 Volumes) by Robert A. Caro In
January of 2016, I picked up my first book in this Caro’s series on
Lyndon Johnson. It wasn’t until June that I finished my fourth, but I
consider finishing all (~3,500 pages) of them to be one of my proudest reading achievements.
If there is one line that sums up the whole series it’s this: It’s that
power doesn’t only corrupt. That’s too simple. What power does is reveal.
It’s also easy to be disillusioned by politics right now but for me,
getting lost in these Lyndon Johnson books has been a helpful and
educational process. Because you learn two things 1) that things have
always been complicated and confusing but they tend to turn out alright
2) that our system, whatever its flaws, can still produce good results
from bad men. And without a doubt, that’s a good reminder to have on the
eve of 2018.
The Last Lion: Winston Spencer Churchill (3 Volumes) by William Manchester Don’t
think an individual can make a difference in history? Then you haven’t
read enough about Winston Churchill. The scenes in this book: warning
about the looming threat of Nazism, the retreat at Dunkirk, persevering
through the Blitz, vowing to fight on the landing grounds and the
beaches and in the streets, whatever the cost may be. The sheer
determination of this man, to take an entire country on your back and
defy a horde which had overrun the European continent in a matter of
months…it’s almost breathtaking to think about. (If you were to only
read one thing in the next
year, you could do a lot worse than either of these series. They contain
dozens of books within them and will teach you about so much more than
just the man they are ostensibly about. Please, please read them.)
Mr. Eternity by Aaron Thier / The Count of Monte Cristo by Alexandre Dumas These
books really have nothing to do with the events of 2016 but they are
long and entertaining and they will make you forget your problems for
the next 12 months. I thought I’d read Monte Cristo as a kid, but
clearly I didn’t. Because the actual book is a 1,200 page epic of some
of the most brilliant, beautiful and complicated storytelling ever put
to paper. What a book! When I typed out my notes (and quotes) after
finishing this book, it ran some 3,000 words. As for Mr. Eternity, it’s a fun, epic jaunt through the distant future (as though it was the past) in the vein of Voltaire’s Candide. Candide isn’t a bad read for this year either: “We must cultivate our own garden.”
Like to Read?
I’ve created a list of 15 books you’ve never heard of that will alter your worldview and help you excel at your career.
BigQuery just announced the ability to cluster tables — which I’ll describe here. If you are looking for massive savings in costs and querying times, this post is for you.
Warning: This post plays with several terabytes of data. Set up your BigQuery cost controls, buckle up, keep calm, and query on.
Can you spot the huge difference between the query on the left and the one on the right?
What can we see here:
Kubernetes was a popular topic during 2017 in Wikipedia — with more than 400,000 views among the English Wikipedia sites.
Kubernetes gets way less pageviews on the mobile Wikipedia site than on the traditional desktop one (90k vs 343k).
But what’s really interesting:
The query on the left was able to process 2.2 TB in 20 seconds. Impressive, but costly.
query on the right went over a similar table — but it got the results
in only 5.4 seconds, and for one tenth of the cost (227 GB).
What’s the secret?
you can tell BigQuery to store your data “sorted” by certain
fields — and when your queries filter over these fields, BigQuery will
be smart enough to only open the matching clusters. In other words,
you’ll get faster and cheaper results.
Previously I explained my lazy loading of Wikipedia views
into partitioned yearly BigQuery tables. As a reminder, just for 2017
we will be querying more than 190 billion of pageviews, over 3,927
Wikipedia sites, covered by over 54 billion rows (2.2 TBs of data per
To move my existing data into a newly clustered table I can use some DDL:
CREATE TABLE `fh-bigquery.wikipedia_v3.pageviews_2017` PARTITION BY DATE(datehour) CLUSTER BY wiki, title OPTIONS( description="Wikipedia pageviews - partitioned by day, clustered by (wiki, title). Contact https://twitter.com/felipehoffa" , require_partition_filter=true ) AS SELECT * FROM `fh-bigquery.wikipedia_v2.pageviews_2017` WHERE datehour > '1990-01-01' # nag
-- 4724.8s elapsed, 2.20 TB processed
Note these options:
CLUSTER BY wiki, title: Whenever people query using the wiki column, BigQuery will optimize these queries. These queries will be optimized even further if the user also filters by title. If the user only filters by title, clustering won’t work, as the order is important (think boxes inside boxes).
This option reminds my users to always add a date filtering clause to
their queries. That’s how I remind them that their queries could be
cheaper if they only query through a fraction of the year.
4724.8s elapsed, 2.20 TB processed:
It took a while to re-materialize and cluster 2.2 TB of data — but the
future savings in querying makes up for it. Note that you could run the
same operation for free with a bq export/load combination — but I like the flexibility that DDL supports. Within theSELECT * FROM I could have filtered and transformed my data, or added more columns just for clustering purposes.
Better clustered: Let’s test some queries
SELECT * LIMIT 1
SELECT * LIMIT 1
is a known BigQuery anti-pattern. You get charged for a full table
scan, while you could have received the same results for free with a
table preview. Let’s try it on our v2 table:
SELECT * FROM `fh-bigquery.wikipedia_v2.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' LIMIT 1
1.7s elapsed, 180 GB processed
Ouch. 180 GB processed — but at least the date partitioning to cover only June worked. But the same in v3, our clustered table:
SELECT * FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' LIMIT 1
1.8s elapsed, 112 MB processed
Yes!!! With a clustered table we can now do a SELECT * LIMIT 1, and it only scanned 0.06% of the data (in this case).
Base query: Barcelona, English, 180->10GB
SELECT wiki, SUM(views) views FROM `fh-bigquery.wikipedia_v2.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' AND wiki LIKE 'en' AND title = 'Barcelona' GROUP BY wiki ORDER BY wiki
18.1s elapsed, 180 GB processed
Any query similar to this will incur costs of 180 GB on our v2 table. Meanwhile in v3:
SELECT wiki, SUM(views) views FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' AND wiki LIKE 'en' AND title = 'Barcelona' GROUP BY wiki ORDER BY wiki
3.5s elapsed, 10.3 GB processed
On a clustered table we get the results on 1/6th of the time, for 5% of the costs.
Don’t forget the clusters order
SELECT wiki, SUM(views) views FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' -- AND wiki = 'en' AND title = 'Barcelona' GROUP BY wiki ORDER BY wiki
3.7s elapsed, 114 GB processed
I don’t make use of my of my clustering strategy (in this case:
filtering by which of the multiples wikis first), then I won’t get all
the possible efficiencies — but it’s still better than before.
Smaller clusters, less data
The English Wikipedia has way more data than the Albanian one. Check how the query behaves when we look only at that one:
SELECT wiki, SUM(views) views FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' AND wiki = 'sq' AND title = 'Barcelona' GROUP BY wiki ORDER BY wiki
2.6s elapsed, 3.83 GB
Clusters can LIKE and REGEX
Let’s query for all the wikis that start with r, and for the titles that match Bar.*na:
SELECT wiki, SUM(views) views FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' AND wiki LIKE 'r%' AND REGEXP_CONTAINS(title, '^Bar.*na') GROUP BY wiki ORDER BY wiki
4.8s elapsed, 14.3 GB processed
That’s nice behavior —clusters can work with LIKE and REGEX expressions, but only when looking for prefixes.
If I look for suffixes instead:
SELECT wiki, SUM(views) views FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' AND wiki LIKE '%m' AND REGEXP_CONTAINS(title, '.*celona') GROUP BY wiki ORDER BY wiki
(5.1s elapsed, 180 GB processed
That takes us back to 180 GB — this query is not using clusters at all.
JOINs and GROUP BYs
This query will scan all data within the time range, regardless of what table I use:
SELECT wiki, title, SUM(views) views FROM `fh-bigquery.wikipedia_v2.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' GROUP BY wiki, title ORDER BY views DESC LIMIT 10
64.8s elapsed, 180 GB processed
a pretty impressive query — we are grouping thru 185 million
combinations of (wiki, title) and picking the top 10. Not a simple task
for most databases, but here it looks like it was. Can clustered tables
improve on this?
SELECT wiki, title, SUM(views) views FROM `fh-bigquery.wikipedia_v3.pageviews_2017` WHERE DATE(datehour) BETWEEN '2017-06-01' AND '2017-06-30' GROUP BY wiki, title ORDER BY views DESC LIMIT 10
22.1 elapsed, 180 GB processed
As you can see here, with v3
the query took one third of the time: As the data is clustered, way
less shuffling is needed to group all combinations before ranking them.
My data can’t be date partitioned, how do I use clustering?
Two alternatives: 1.
Use ingestion time partitioned table with clustering on the fields of
your interest. This is the preferred mechanism if you have > ~10GB of
data per day. 2. If you have smaller amounts of data per day, use a
column partitioned table with clustering, partitioned on a “fake” date
optional column. Just use the value NULL for it (or leave it
unspecified, and BigQuery will assume it is NULL). Specify the
clustering columns of interest.
tip: I’ve heard that for now it’s better if you load data into your
clustered tables once per day. On that tip, I’ll keep loading the hourly
updates to v2, and insert the results into v3 once the day is over. From the docs: “Over
time, as more and more operations modify a table, the degree to which
the data is sorted begins to weaken, and the table becomes partially
Clustered tables in only one of many new BigQuery capabilities: Check out GIS support (geographical), and BQML capabilities (machine learning within BigQuery).
In my book (Data Science on the Google Cloud Platform), I walk through a flight-delay prediction problem and show how to address it using a variety of tools including Spark Mlib and TensorFlow. Now that BigQuery ML has been announced, I thought I’d show how to predict flight delays using BQ ML.
Make no mistake — you still have to collect the data, explore it, clean it up, and enrich it. Essentially all the stuff I do in Chapter 1–9. In Chapter 10, I used TensorFlow. In this article, I will use BQML.
Create Regression Model
Here’s a BigQuery query to create the model:
#standardsql CREATE OR REPLACE MODEL flights.arrdelay
OPTIONS (model_type='linear_reg', input_label_cols=['arr_delay']) AS
SELECT arr_delay, carrier, origin, dest, dep_delay, taxi_out, distance FROM `cloud-training-demos.flights.tzcorr` WHERE arr_delay IS NOT NULL
starts with a CREATE MODEL, with the name of the model looking just
like a table name. Note: ‘flights’ is the name of the dataset to store
the resulting model — so, you’ll need to create an empty dataset before
running the query.
The options specify the algorithm — in this case, a linear regression algorithm, with arr_delay being the label
Essentially, I’m pulling in the predictor and label variables in the SELECT
About 10 minutes later, the model is trained and evaluation results have been populated for each iteration:
The loss here is mean squared error, so the model converges on iteration #6 with a RMSE of about sqrt(97) = 10 minutes.
Predict using Model
The purpose of training a model is to predict with it. You can do model predictions with a SQL statement:
#standardsql SELECT * FROM ML.PREDICT(MODEL flights.arrdelay, ( SELECT carrier, origin, dest, dep_delay, taxi_out, distance, arr_delay AS actual_arr_delay FROM `cloud-training-demos.flights.tzcorr` WHERE arr_delay IS NOT NULL LIMIT 10))
This results in:
you can see, because we trained the model to predict a variable called
“arr_delay”, ML.PREDICT creates a result column named
predicted_arr_delay. In this case, I’m pulling 10 rows from the original
table and predicting the arrival delay for those flights.
Create Classification Model
the book, I don’t actually try to predict the arrival delay as such.
Instead, I predict the probability that a flight will be more than 15
minutes late. This is a classification problem, and you can do that by
changing the training query slightly:
#standardsql CREATE OR REPLACE MODEL flights.ontime OPTIONS (model_type='logistic_reg', input_label_cols=['on_time']) AS
SELECT IF(arr_delay < 15, 1, 0) AS on_time, carrier, origin, dest, dep_delay, taxi_out, distance FROM `cloud-training-demos.flights.tzcorr` WHERE arr_delay IS NOT NULL
Here’s the evaluation results:
and an example of the predictions:
Evaluating the model
is possible to evaluate the model on an independent dataset. I don’t
have one handy, so I’ll just show you how to run the evaluation on the
same dataset the model was trained on:
#standardsql SELECT * FROM ML.EVALUATE(MODEL flights.ontime, ( SELECT IF(arr_delay < 15, 1, 0) AS on_time, carrier, origin, dest, dep_delay, taxi_out, distance FROM `cloud-training-demos.flights.tzcorr` WHERE arr_delay IS NOT NULL ))
Discussions about Facebook’s data privacy and corporate power are about to extend to money. Grab the popcorn!
By Caitlin Long
What does Facebook’s new cryptocurrency mean? According to news reports, Facebook will shortly launch Libra,
a global cryptocurrency available to users of its suite of platforms
(including Messenger and WhatsApp). Presumably any merchant with an
account on these platforms could transact in the cryptocurrency with
customers who also have accounts — for anything, such as online
purchases, and physical-world purchases such as groceries and
restaurants. Facebook this week revealed plans to announce details on June 18 and confirmed its cryptocurrency will be a “stablecoin”
whose value will be tied to a basket of fiat currencies. Based on
Facebook’s statement and several anonymous comments made by people tied
to the project in interviews with The Information, here are my six predictions.
1. Facebook’s cryptocurrency will be a powerful force for good in developing countries, which is where Facebook intends to market the product
Because central banks in developing countries are notorious for their
lack of discipline in maintaining the value of their fiat currencies,
which too often lose purchasing power. The best example among many is Venezuela,
which is experiencing hyperinflation worse than that of Germany after
World War I. By providing citizens of developing nations with access to a
store-of-value that is more reliable than their government-backed
currencies, Facebook’s cryptocurrency will indirectly exert fiscal and
monetary discipline on developing nations — which will improve the lives
of many people globally.
Facebook will pay interest to holders of its cryptocurrency, and this
will eventually lead to populist calls to repeal corporate subsidies to
banks at the heart of the US banking system
go out on a limb and predict Facebook will pay interest to users of its
cryptocurrency. Why? Because the assets backing the cryptocurrency will
generate interest income (especially if, according to some reports,
that basket includes “low-risk securities”).
If Facebook doesn’t share these interest spoils with users, a chorus of
critics will loudly publicize how much money Facebook and its partners
the magnitude of the interest income at play? If Facebook parks the
entire US dollar balance at the Federal Reserve via one of its bank
partners, for example, it could earn 2.35% risk-free — that’s
$235 million for every $10 billion deposited into its cryptocurrency.
These profits will quickly turn into a new hot potato for Facebook
politically, if not shared with users.
there’s a side benefit — the brouhaha this issue could create would
reveal the magnitude of corporate welfare at the heart of the US banking
system. The 2.35% number is the actual interest rate the Fed pays its
member banks for interest on excess reserves (IOER) — and this year it’s
projected to amount to $36 billion of corporate welfare paid to US banks,
which equates to roughly half the amount the US spends on its food
stamp program. Just imagine how critics will have a field day shouting
“corporate welfare for Facebook” if Facebook and its partners simply
pocket that amount.
true that other stablecoin issuers almost always pocket the float
rather than sharing it with their customers. But Facebook’s stablecoin
will probably be too big and visible to get away with this — so it’s
unlikely to be able to sweep the issue under the rug. That’s a good
segue to the next point.
3. Facebook’s foundation will grow to garner big power in global capital markets
Facebook plans to cede governance control of its project to an independent foundation,
which it recently formed in Switzerland. This is a positive — not only
does it give Facebook defense against antitrust allegations, but it also
helps reduce the degree of its cryptocurrency’s centralization. This
foundation is likely to become a huge power within global capital
markets relatively quickly — because it will do what central banks do,
which is to define the basket weights for the fiat currencies to which
the stablecoin is pegged and manage the assets to ensure the peg doesn’t
break. There are plenty of powerful “basket-setters” in capital
markets, and their power to move markets can be significant — think of
the committee that defines components of the Dow Jones Industrial
Average Index (DJIA) or the S&P 500 Index, or central banks that peg
their currencies to baskets (such as China’s PBOC).
these lines, at first blush it struck me as backwards that Facebook is
selling the right to participate in its network for $10 million each,
since usually the transaction processers (“miners”) in cryptocurrency
markets are paid for their services rather than need to pay-to-play.
But, remember #2 — there’s a big pot of interest income for all of them
to divide among themselves, especially if they don’t pay interest to
users, and a lot of foreign exchange trading volume is at stake. No
wonder why dozens of banks will potentially be involved.
4. Facebook will face regulatory uncertainty — which will shed light on many outdated financial regulations in the process
Facebook’s cryptocurrency a security? If it is, will users face the
absurdity of needing a US brokerage account to buy a cup of coffee with
it? Will Facebook catch breaks from regulators that smaller start-ups
haven’t — because of the tax data honeypot Facebook’s project will
generate for governments? That’s a good segue to the next prediction.
5. Facebook’s regulatory reporting program will open all kinds of interesting discussions
We may be about to find out how many of Facebook’s 2.3 billion users
are real — since users of its cryptocurrency would need to prove their
identity and pass know-your-customer compliance requirements. The Information
story noted Facebook “plans to provide more stringent forms of identity
verification and fraud detection than do most cryptocurrencies.”
But there’s more. Discussions about Facebook’s data privacy and corporate power are about to extend to money. Grab the popcorn!
will open all kinds of conversations about the extent of data privacy,
financial privacy, overseas asset reporting and tax compliance and
reporting burdens — and could potentially challenge the
extra-territorial reporting requirements imposed by the US government on
non-US businesses. Governments everywhere will view Facebook’s
cryptocurrency as a huge honeypot of data about how users spend
money — with all the privacy and tax reporting implications that data
honeypot entails, because every transaction would be traceable by
governments. This would give certain officials what they’re hoping for,
which is the ability to trace, monitor and analyze every dollar spent
rather than piecing together piecemeal reports (currently, banks file
suspicious activity reports for transactions above $10,000). Here’s one
example in a statement made during a May 2019 speech by Sigal Mandelker, Under Secretary for Terrorism and Financial Intelligence:
When FinCEN analyzed millions of dollars of remittance transactions with suspected links to terrorism, it found they averaged less than $600 each.
In an era where a radicalized suicide bomber can bring a tragic end to
the lives of hundreds for nothing more than the price of duct tape, a
vest, and supplies, we cannot afford to allow any money to flow to terrorists.(emphasis added)
Moreover, these reporting requirements are about to grow substantially once the Financial Action Task Force recommends that all financial transactions (including cryptocurrencies) embed data about the beneficial owner in each transaction, for both the sender and receiver.
As I’ve explained in a previous Forbes.com post, I doubt the data dragnet required by US reporting requirements would survive a constitutional challenge. How interesting if Facebook were at the center of such a challenge. Would it fight or cooperate??
6. Facebook’s cryptocurrency will turn out, in the end, to be a Trojan horse that benefits bitcoin
my biggest prediction: Facebook’s foray into cryptocurrency will end up
benefiting bitcoin. It will take time, but Facebook will greatly
accelerate the pace of teaching people about cryptocurrencies. And when
this happens, more people will turn to bitcoin for one simple
reason — bitcoin is scarce, while Facebook’s cryptocurrency is not.
People will migrate over time to the most honest ledger for storing
their hard-earned wealth — and that’s not fiat currencies or derivatives
thereof, including Facebook’s cryptocurrency.
phenomenon actually happened in Venezuela, as early bitcoiner Nick
Spanos recently pointed out to me. When the Maduro regime introduced the
ill-fated petro cryptocurrency, the government made a concerted effort
to educate Venezuelans about cryptocurrencies — and it correlated to a spike in bitcoin use by Venezuelans.
foray into cryptocurrency will likely end up being a beneficial detour
on the path to broader bitcoin adoption. Bring it on!
Long is a 22-year Wall Street veteran who has been active in bitcoin
since 2012, and whose passion is a fair and stable financial system.