A Winding Road to InfoSec

Index Link to heading

Introduction Link to heading

Over the last three years my professional and personal life has dramatically changed for the better. I hope it encourages you to seek meaningful endeavors with your life and the tough personal changes that are often required to do something of measure.

Bomb Threat Link to heading

Three years ago, in October of 2017, I was assigned the job of tracking down a significant bomb threat. This was a difficult task because it involved children. And there was a critical moment in the investigation where I held the entirety of a student’s future in my hands.

In one hand there existed a reality where the student landed in the Federal court system with little support. In the other hand, a space where someone took on the role of advocate for this student and tried their very best to understand and speak on the student’s behalf.

If you have never held someone’s future like this, where you are the beginning, the middle, and the end, it is quite a position to be in. For me, I was convicted in how I should act and I chose the latter hand which was the harder path. That choice was a incredibly difficult endeavor but one that I’m proud to have been given the opportunity to help with.

Two things though that I’ve not shared publicly.

I asked and was given permission to meet with the student and their principal so that I could encourage the student. This was an emotional and meaningful encounter and to me, it was the right thing to do. To build this child up and let them know that even when they felt alone, they were not.

And also, my community made me “Honorary Captain of the Football Team” and celebrated me that evening at a local game. To be on the football field under the lights at a night game and have the stadium stands full of parents and peers, and to see them rise to their feet to recognize one’s good work is quite a revelation. At least it was for me. I hope and wish that everyone could get a chance to be recognized like this. It is such a rare thing and it is a gift.

This event is chronicled in this article and on Ayman’s podcast.

Software Vulnerabilities Link to heading

I had felt something change inside me in October and I was struggling to understand how I was changing. Around this time, in December of 2017, I was performing some routine maintenance on a massive piece of software. I noticed something odd. And generally, where I may have been tempted to respond with apathy, this change taking place in me convicted me that I should really dig into this oddity.

This led me to finding quite a large vulnerability in the software. I wrote a proof-of-concept exploit in Go, compiled it instantly to all major OS platforms, and started a communication with the vendor that included this proof-of-concept.

“Oh yes, we’ve fixed this vulnerability and you can find the patch on the portal.”

This response struck me as odd. My organization I was sure had never received a notification or email.

I knew that this software had a large community, so I decided to reach out to this group and introduce myself and get their opinion. What we discovered was even more unsettling. No one had ever received notification of this vulnerability and because of this, no customer had ever installed the patch.

I responded to the vendor and asked when this patch had been released.

“I believe it was sometime last month.”

I waded through their portal to find the bug fix and after downloading it, I simply ran the file command to inspect the meta data.

Fourteen months. That was the original creation date in the pdf along with the author’s name.

This knowledge upset many, and the vendor downplayed the significance. I warned my new friends that they should not expose this software to the public internet but there was very little leverage for them to present to their leadership.

Confronted with this, I knew I couldn’t sit idly, so I got to work.

I tracked down three more severe vulnerabilities over the next three weeks. To confirm the most severe, I wrote a proof-of-concept, again in Go, and sent the binary to a Sys Admin peer. He aimed this binary at his endpoint and ran it, and with zero authentication or authorization it was harvesting the W2s of his entire organization…for the entire history of said organization.

This affirmed that with a slight modification, I could perform a drive by attack and target thousands of Federal, State, and Educational entities and harvest their PII data with no one being the wiser.

This was a big deal.

Myself and my small group of peers and friends informed the vendor and things were tense. My organization had never placed this software on the open internet. But my buddy who I sent the proof-of-concept to, after seeing the potential for damage and loss, decided that this sounded like a good plan and convinced his organization to remove their instance from the public web too.

We encouraged the vendor to be truthful with their customers and inform them. But we also agreed to an embargo, so we waited.

Meanwhile a large conference for this software took place in the spring of 2018, which many people attended. Everything was going normally at this conference until a panel session, which my buddy was on.

In a roomful of hundreds of people and leaders in this space, someone stood up and asked the following question of my buddy, who is well respected in this community.

“Do you have any thoughts on last years security vulnerabilities? Has this impacted your organization in any way?”

Not wanting to break the embargo but not wanting to lie he replied

“As of now, my organization has removed their instance from the public internet.”

Since I did not attend, I will relay what he described to me.

“It was absolute pandemonium. Almost as if you had suddenly opened a door to the trading room floor on Wall St. System Admins were ripping open their bags, connecting to their networks, and shutting down systems. IT Managers were yelling into their phones ‘Shut it down, shut it down.’”

This scene, described, would really influence my thinking in the coming months. Primarily as I continued to find vulnerabilities in this software and the vendor continued to try to not communicate them effectively.

In the Summer of 2018, after finding another SQLi, they released a statement and patch about this issue. It included a full description, the potential impact, and was the first time I was recognized publicly for my work.

We also created a non-profit, OpsecEdu, to organize our security research for the future.

In October of 2018 my new friends invited me to their annual conference and offered to pay for my stay if I spoke. I asked my wife if there was anything I could do for her while I attended.

“Would you mind not drinking any alcohol while you are down there?”

I agreed.

Alcoholism Link to heading

I attended the conference and everywhere I went, after I was introduced, someone would exclaim “Oh you’re Jared! Let me buy you a drink!”

I can’t tell you how many times I was confronted in this well spirited offer. Inside me though, I was reflecting on my relationship with alcohol. This trip, being 8 days long, meant this would be the longest I had gone without drinking for 15 years. And being honest, I was tempted. There were some top shelf offers that had me really wishing in that moment that I had never agreed.

When I got home, I was proud that I had fulfilled my promise to my spouse. And that night to celebrate, I bought myself some microbrews and started to celebrate and relax and drink.

But something was bugging me.

Opening up my laptop, I ran a small report tracking my alcohol spend for the last 10 years. I saw a graph 📈 that was steadily rising to the right. I saw some weird rises and dips in the trend line. So, I drilled down a bit and selected only 2017 and looked at the per month alcohol spend. And this was even more revealing.

Looking closely I noticed that my spend was doubling for the months of March and June and I knew instantly why. These are the birth month and death month of my kid brother. I knew I was coping using alcohol. I was remembering the times that my wife had asked me if maybe I shouldn’t drink so much. Or had expressed the thought that maybe alcohol was having a negative impact on my life and my relationships. And I was convicted.

I went downstairs and expressed to my wife “I’d like to try not drinking anymore.”

The next day, October 29th 2018 I asked my Uncle to sponsor me.

After thirty days of not drinking. My wife and I agreed that I could tell the kids. This took my commitment to another level. I knew I was in it for the long haul at that point. And I was scared I was going to mess up.

Trust as the Ultimate Currency Link to heading

Confronted with the new reality of not drinking, I can’t lie, it was incredibly tough. But that change that I believe had started in 2017 was still happening. I txt’d or called my Uncle every day, multiple times a day, for the first 90 days. But I also started to think about that Vendor who had concealed their actions and how much trust they had lost.

Another commitment I made was that every day I was going to try and think about the trust of my family and friends as a tangible asset. And whenever confronted with a choice that would impact their trust, I would visualize it as if holding their trust in my hands and I would ask “Is the choice I’m about to make going to lose their trust?”

One day I was struggling, and I remember being angry that I was my own enemy and that I didn’t know how I could overcome and forgive myself. I thought about that for a bit and realized that I needed to love my enemy.

I googled those terms “Love my enemy.” And the first thing that hit was Dr. King’s speech “Love your enemies.” I listened to it then. And I proceeded to listen to it daily.

I was still helping with OpsecEdu and finding vulnerabilities during this period when a disclosure went sideways. This new vendor didn’t appreciate the work at all. And they got real mean, real fast. Many of my peers felt I was justified in going public and shaming this vendor.

Around February of 2019 I encountered Marcus Carey online. Out of the blue he asked me to call him. And he spent over an hour encouraging me and asking what I wanted to accomplish and if shaming the vendor with my project was going to help.

I got off the phone and I felt like drinking. I was stressed. And so, I decided to listen to Dr. King’s speech again. And that is when things started to click.

“When the opportunity presents itself for you to defeat your enemy, that is the time which you must not do it.”

InfoSec has a long history of debating on what disclosure formats are the best. Whether it be name and shame, full disclosure, coordinated disclosure, or even no disclosure. I looked at the history of the last 20 years to try to understand what type was the most effective. But not from the vantage of what would get the vulnerability remediated the quickest. But rather if there was a way to convince the organization to change their system. And that’s when Dr. King got me again.

“It is the refusal to defeat any individual. When you rise to the level of love, of its great beauty and power, you seek only to defeat evil systems. Individuals who happen to be caught up in that system, you love, but you seek to defeat the system.”

For me, I knew that I was not going to shame this person. This person inside of this organization inside of this system. Because to do that would be to hate them. And I knew that hating them was leading me to being angry at myself which was pushing me to want to drink.

So with a brief email, I parted ways with this vendor. And you know what, I felt and still feel a lot of peace about that.

Community Link to heading

In the spring of 2019 Marcus and Jen asked me if I wanted to be in their book Tribe of Hackers: Red Team. I was stoked. I don’t feel like I fit in most days and here someone was listening to me speak technically and was telling me that I belong. I was being included by someone in their group and that felt good.

I also started hacking on an idea. I realized that all the systems I’m seeing have vulnerabilities. It is pervasive. There is no end. And with this knowledge I considered that maybe for now, finding them and remediating them isn’t the best use of my time. Since I can code too, I started to hack on a project that I believe will help my community.

Nathan, April, and I, leading OpsecEdu, doubled down and started to seek ways to better communicate with all of its members. We started a Slack channel in the Summer of 2019 which seems to be the best method for us to share. The community trusts each other and actively seeks to help and share what they know. It has been super affirming.

In the Fall of 2019 I get a bunch of my new friends from Tribe of Hackers to agree to help kids. We called it Bring Your Ethical Hacker to School Day and scheduled virtual events where contributors from the book agreed to meet with a K12 teacher and their classroom. The entire event was amazing, and we reached classrooms as far away as Australia. A few mentions are found below.

Life moved forward and I was lucky that I was able to reveal my project, Kushtaka at BsidesPDX 2019. Kushtaka is an all-in-one honeypot and honeytoken sensor system built for easy installation and maintenance. I gave a lightning talk about it. I turned right around and was given the opportunity to offer a full 60 minute talk about my project at BsidesBoise, titled “Defense through Deception.” The feedback was excellent, and I felt really affirmed.

Then a honeypot/canary sensor allegedly detected an anomaly on a large corporate network inside a school district. A device of unknown origin was found supposedly scanning their infrastructure on a Saturday.

And just the other night, Nathan released a PowerShell script that vastly simplifies Kusthaka’s installation on Windows. And before I knew it, April had messaged us that she had spun up a full Kushtaka stack including a sensor.

One thing that I never considered with Kushtaka. In the case of education, Kushtaka has the ability to help detect curious students early, before their curiosity leads them to a choice of legal consequence. This means that students can have softer landings as law enforcement may not need to be involved if no laws were broken. As Kushtaka continues to grow, this will be a win for both IT departments and Students.

Conclusion Link to heading

By giving up alcohol my relationships with my wife, my kids, my peers, and my family has never been better.

By helping a student in my own unique way, I’ve earned the trust of many in my local community.

By taking a risk to help report software vulnerabilities to a vendor, I know that Nathan McNulty, April Mardock, Nick Vissari, Jessy Irwin, Jay Lagorio, Doug Levin, and Sean O’Brien and many others have built something meaningful in OpsecEdu. Something that is helping others.

And by choosing not to crush someone inside of a system, I’m now part of a Tribe. One that includes Marcus Carey, Jennfier Jin, Kim Crawley, Phillip Wylie, Wirefall, Edward Prevost, Ming Chow, Keirsten Brager, Isaiah Sarju, Emily Crose, Robert Walker, Steve Ragan, Ian Anderson, Mr. Jeff Man, Paul W. Brager Jr, Davi Ottenheimer, Terence Jackson, Kyle Bubp, and Rob Fuller.

Lately people ask me what advice I have for getting into InfoSec, which led me to writing this post. I am convinced that the following is what matters most.

“Seek to earn the trust of people you love and respect, and then work to keep it.”

I hope this helps you,

Jared