O’Reilly Security Conference

Disclaimer: I was a speaker at the conference. As such, O’Reilly Media covered most of my travel expenses, as well as provided me with a Speaker pass. If you think such benefits, nice though they were, had a significant impact on my opinion, to me it just means we don’t know each other very well yet. Trust me when I say that they do NOT… Happy to discuss as needed…

TL;DR: The experience of being part of the inaugural O’Reilly Security Conference was amazing. The content I watched was excellent, the venue/logistics worked really well, and I really liked the “vibe” on the conference. 10/10!


Source: O’Reilly Media – click for license details.

This longish post is about my experience on the O’Reilly Security Conference. I summarize what I learned from each session I attended, as well as general opinions. I can’t think in prose, so this is mostly in list format. Without further ado:

Format and Venue

  • 4-track conference, held at the New York Hilton Midtown.
  • Pre-conference training and tutorials, an Ignite session, then 2 days with morning keynotes followed by morning and afternoon sessions.
  • Good breaks in between sessions (ranging from 15 minutes to 1 hour)
  • No idea on attendance, likely in the mid hundreds.

Tutorials and Ignite

I attended Jim Manico’s half-day tutorial on “Modern IdM” hoping to learn more about Web authentication and I was not disappointed. He covered OAuth in detail, as well as session management, and recommendations around password storage. He’s a very energetic and engaging speaker, and time flew by.

The afternoon was reserved for the Apache Drill tutorial led by Charles Givre, from Booz Allen Hamilton. Charles took us through the rationale for Apache Drill – basically a SQL-supporting unifying front-end for disparate back-end data stores – and led exercises on data manipulation. Drill can be a fantastic tool for a data scientist to easily get at disparate data sources.  I’m a SQL newbie and struggled with some of the exercises, but that is on me and not on the tutorial. He also based the exercises on a pre-configured VM that has other data science tools. This will come in very handy…

In the evening, Jerry Bell and Andrew Kalat hosted the Ignite talks (lightning fast talks with auto-advancing slides). Jerry and Andrew host the Defensive Security podcast , probably my favourite security podcast. It was a privilege to chat with them. The talks were interesting, ranging from the need to shy away from hero-focused security work, to how we can do better at training/education, to the use of existing intelligence/data sources. Great talks, easy-going format.


Then there was… karaoke… For those that are not familiar, “slide karaoke” is a fun-filled/terrifying (depending on your point of view) format where someone is presented random slides at a fixed-time interval and the they have to “improv” their way to a somewhat coherent talk structure. Andrew and Jerry asked for 5 volunteers…. and I was one of them….

I don’t quite remember what all my slides were, but there were references to llamas, some sort of potato-based disease, and rule breaking. 🙂  I’m just hoping I made it entertaining for the audience…

Lesson learned: Courtney Nash is a master at this: she was funny, coherent, engaging, … She’s a very tough act to follow, which just happened to be my spot in the roster… You have been warned 🙂

Seriously, though: it was great fun, and I hope others join in. It was a great environment, people were having fun, and part of being in this industry is this sense of community that we build. It was a privilege to be able to take part in that.


On day 1, following the intro from Allison Miller and Courtney Nash, Heather Adkins from Google kicked things off by showing us how some of the main classes of security incidents – be they insecure defaults, massive theft, or instability – have been happening in different forms since the 1980s. After pointing to the increased siloization(sp?) of our industry as a possible cause, she urged us to think about broader platforms, and to design with a much longer timeframe in mind.

Richard Thieme took us through a sobering view of the psychological challenges in our career. Drawing parallels to the intelligence community and the challenges faced there, Richard rightfully reminded us to stay mindful of our needs as individuals and building adequate support networks in our lives.

Becky Bace did a great job of comparing the challenges of infosec today with the early days of the auto industry, and how we can use some of the lessons learned there to improve it. Given my interest in economics and incentives, I was silently clapping pretty much all the time.

Unfortunately I missed most of the day 2 keynotes – I look forward to watching video later. What I did catch was the latter part of Cory Doctorow‘s impassioned and cogent plea for more involvement from us as individuals into the immensely important debate about the very nature of property and democracy in modern society. There are key discussions and precedent-setting court cases taking place now, and many of the key societal instruments we take for granted are at risk.

Day 1 Sessions

Speak Security and Enter. Jesse Irwin led a great session focused on how to better engage with users when it comes to discussing security and privacy. She laid out very well defined steps for improving. If I could summarize her session in one idea would be: have more empathy to your user community. From using relatable examples, to framing the issue positively or negatively, and many other suggestions. Hearing her tell of the adventure of teaching security to 8-year-olds was priceless!

Notes from securing Android. Adrian Ludwig from the Google Android team took us through a data-driven journey into the Android security ecosystem. After reminding us that Android security must accommodate from $20 phones to modified units used by world leaders, Adrian focused on three aspects: active protections made by the Google ecosystem, options available for enterprise decisions (such as allowing external app stores or not), and details about the Android OS itself. He made a very compelling case that the security architecture of a modern Android-powered device such as the Google Pixel rivals what other options exist in the mobile ecosystem (iOS, WindowsPhone). This was one of the best talks I attended.

Groupthink. Laura Mather has had a very interesting career, including time at the NSA, eBay, founding SilverTail (where I had the pleasure of working for her), now leading Unitive. Her talk was not a ‘security’ talk, but rather a look into the issue of groupthink, often caused by unconscious biases. Fundamentally, the variety of challenges in modern security environment should be met by having a diverse workforce generate ideas based on diverse points of view. In order to achieve this, we need to work on the issue of lack of diversity. Laura pointed out specific ways to avoid unconscious bias in hiring, particularly being aware of, as an interviewer/hiring manager, not looking for someone “just like me”. Hiring decisions should be matched on values, not on superfluous characteristics that lead to biased outcomes.

UX of Security Software. Audrey Crane leads a design firm, and made the case for proper UX design taking into account the people who will actually use the product. Her firm conducted research into usage habits related to SOC roles, and came up with a few personas (different from the typical ‘marketing’ personas) and then showed an interface design that takes those personas into account. Her recommendations are for vendors to take this aspect of the product creation process seriously, and for buyers of software to not only demand better software from a usability perspective, but to actively try out any software being purchased with the intended audience.

Social Scientist. Andrea Limbago brought a “social scientist” perspective to the broad issues around information security. She framed the discussion in terms of Human Elements, Geopolitical trends, and Data-Driven Security. The human elements section looked at an structure-agent dynamic (top-down versus behavioural) and advocated approaches to evolving the security subculture. Very interesting, as were the comments around security still having a cold war framework, and that there is a gap on the usage of data within security conversations.

Day 2 Sessions

Are we out of the woods?. Kelly Harrington from the Google Chrome team talked about Web security issues. She covered some key issues – how updates are not universal, how older devices get attacked, and the scourge of what Google calls Unwanted Software – and delved into details about the exploit kits (Angler, Rig, and others), trends of attacks on routers, plus examples of malicious behaviour by Unwanted Software. She wrapped up by sharing a little about what Google’s Safe Browsing API does and by giving actionable advice on web security. This was a great talk to complement the one on Android security. Finally, extra points for her for the Jane Austen references… 🙂

Criminal Cost Modelling. Chris Baker – a data scientist at Dyn – took us through a whirlwind tour of some underground markets and the actual data he found there for pricing stolen goods, exploit kits, or DDOS services. It was refreshing to see someone dive beyond “oh, underground markets exist” into actual markets, prices, goods, and the possible economic issues that exist in those markets. I loved this session. If there was one session I wish could have been longer, it is this one. I’ll be watching the video when it comes out, many times over.

Economics of CyberSecurity. This session was delivered by yours truly. Happy to announce that slides are available here. I focused on how a brief understanding of economic concepts – Marginal Cost of Information Goods, Information Asymmetry, Externalities, and concepts from Behaviour Economics – can help us rethink some of the broad challenges we face. I hope the audience liked it. I was happy with my delivery and did pick up on a few things I want to improve. I really hope to have the opportunity to keep bringing this message to others.

No Single Answer. Nick Merker – now a lawyer but formerly an infosec professional – and Mark Stanislav – now a security officer with experience as security consultant – focused on cyber insurance. Their session went into the difference between first-party and third-party insurance, then delved into the details of what cyber insurance options exist, what they typically cover (or not), and how these products are currently priced and sold. They also covered some misconceptions around the role of insurance in a risk management program, how infosec should play a role when purchasing cyber insurance products, and how a well-defined and executed security program can help with insurance premiums. I learned a ton, and really liked the session.


The sponsor area was relatively small (maybe 10-15 sponsors total) but the people I spoke to were knowledgeable and the selection was varied. Not so much your typical security vendor, but more those offering solutions that fit into a more modern architecture view of security. There were options for web app security, container security, source code security, etc… I did not focus much on it, given my role as individual contributor.

The conference schedule and details were available via the O’Reilly app (iOS and Android) and things worked well. One suggestion I have is that the app could offer a toggle for ‘hide past events’ on the Full Schedule view, as that would help people choose their next sessions without having to scroll around so much…

Food options during the breaks were varied and quite nice. I loved that we had sushi available on one of the food stations.

As a Speaker

My “field report” would not be complete without a comment about my experience proposing the talk and later as a speaker.

The submission process was well defined, the guidelines for what should go in the submission were clear, and the timelines were very fair. I followed the process via the website and the questions I asked the speaker management team were answered promptly and efficiently. Major thanks to Audra Montenegro (no relation) and her team.

The organizing committee has been very transparent about what their side of the selection process was like. This is tremendously insightful and helpful for future proposals. I particularly liked the use of blind reviews. Blind reviews help us as an industry increase the quality of the content that makes it into the stage, AND increase the chance of hearing from a more diverse pool of contributors. What’s not to like?

Prior to the event, I was able to connect with Courtney Allen and we collaborated on a short email-based interview (which you can find here). She was fantastic to work with and has a keen insight into the role that O’Reilly Media can play in the security landscape.

Bottom line is: If you have defensive-focused security content you want to present, you’re open to be being evaluated on the merits of your content, and want to work with great people putting it together, O’Reilly Security should definitely be on your short list of conferences to submit to.


SIRAcon, day 1

I was extremely fortunate to be able to attend my first SIRAcon last week: it’s not often that one of those ‘aspirational’ conferences was happening at just the right time (found a way to fit in my schedule), not too far from home (Toronto to Detroit is not too far a drive), and was affordable (working on a tight budget here…).

It was a fantastic experience. Many, many thanks to the hosts (Quicken Loans), sponsors (CBI, RiskLens, BT, and BitSight), organizers (David Musselwhite and team), … The venue was great, and it was wonderful to see how the team is proud of Detroit and the turnaround that is happening.

My plan is to have a quick summary of the sessions and then, later, more general comments. There was a decent amount of live tweeting (spread between three hashtags: #SIRAcon2015, #SIRAcon15, and #SIRAcon) , but I thought a quick summary of each session would be a nice idea too.

Warning: my ‘starstruckness’ was out in full force. Totally justified 🙂


Keynote: Douglas Hubbard (@hdr_frm) and Richard Seiersen (@RichardSeiersen)

Doug and Richard opened up SIRAcon with a tour-de-force on applying quantitative methods to Risk analysis. They presented interesting findings showing that an appreciation of qualitative methods seems to be correlated with less comfort/familiarity with statistics concepts. To me, this presents a fantastic opportunity to pursue better dialogue through education 🙂bear

I loved the message that ‘we don’t have enough data’ is not an excuse. They presented a good case for using the beta distribution as a stepping stone from a world of ‘no data’ (where the uniform distribution applies) to a scenario where data is available.

Oh, bonus points for Latinizing the [in]famous bear analogy as ‘Exsupero Ursus‘ 🙂


Jay Jacobs (@jayjacobs) and Tom Montroy (@TomMontroy)

Jay presented an interesting concept of Information Security as a ‘Wicked Problem’ and presented the Cynefin Framework as a basis for discussion on how complex the discussion around good/best/current/… practice applies to our problem space.

Later, Jay and Tom presented several interest exploratory data visualizations looking into how SSL/TLS practices correlate with botnet activity, as well as how indicators such as BItTorrent traffic appear related to Botnet activity and breaches.

I think it was a perfect example of how a data-driven approach to security can lead to insights we would not otherwise have.


J. Wolfgang Goerlich (@jwgoerlich) covered the topic of Culture and the relation to Risk, something he’s been deeply involved in. He collaborates with Kai Roer (@kairoer) on the excellent Security Culture Framework. There were several good examples of how changing user behaviour led to successful outcomes: security awareness training, SDLC, DLP, and physical security. More than that, though, he emphasized the importance of proper feedback loops when addressing culture changes, as well as what I thought was one of the most important messages: culture changes “one conversation at a time”.


Barton Yadlowski (@bmorphism) is an applied mathematician at HurricaneLabs, and presented an introduction and the case for leveraging machine learning in InfoSec, leveraging examples with Splunk, scikit-learn and Spark. He showed how tools such as Splunk can help with unstructured information and normalization, followed by exploratory data analysis. From there, he had an interesting introduction of broad Machine Learning topics and how it can be used to detect anomalies in different scenarios.

It’s always nice to start putting together the description of methods floating around with more practical applications.


Karl Schimmeck (@kschimmeck) covered an effort by SIFMA (Securities Industry and Financial Markets Association, an industry association of 300+ financial services firms) to simplify the process of performing 3rd-party risk assessments. This is extremely important to reduce to compliance costs for both financial services and vendors alike, and hopefully will be adopted by the regulators and the auditing organizations. Using SharedAssessments and SOC2 as initial guidelines, then mapping specific custom requirements and later mapping to NIST-CF, it looks very promising.

As someone who has been on the receiving end of those questionnaires, I really(!) look forward to this effort being successful.


Jack Whitsitt (@sintixerr)  led us down a different path. Drawing on his broad experience and recent activities well beyond typical InfoSec, he urged us all to consider the much broader environment in which InfoSec exists. There’s fundamental issues at multiple levels of abstraction – from individual all the way to global – and, when it comes to organizations, how can we deal with (and support) InfoSec teams being thrown in the middle of geopolitical conflicts?

I loved the talk, but I would like us to explore better the assumption that things are getting worse: are we being affected by the availability bias of all the breaches? That’s an open question (to me, at least).


Thomas Lee from Vivo Security stayed consistent with the ‘quantitative’ theme for SIRAcon and looked at some interesting correlations on factors that may be related to breaches/compromise. He then made a strong case for adopting a more ‘actuarial’ approach to security programs, by taking a better look at loss data as a method of selecting security controls. He then presented an example of applying this methodology to a mid-sized pharmaceutical company, showing how a performing an endpoint update was actually a great approach of reducing impact from phishing.

Personally, I think the approach has merit, as long as we can avoid the trap of spurious correlations. I would have liked to have seen more confidence intervals there too 🙂


Michael Roytman (@mroytman) needs no introductions. His talk brought together concepts that have been around us for a while, coming from the likes of Schneier, Geer, Hutton, Ed Bellis, and others in a discussion of the interplay between Metrics, Data, and Automation. He clearly demonstrated how attackers are able to leverage automation in attacks much better than defenders are able to do so for defense. He also gave a great example of how better datasets can fundamentally change the whole ecosystem: Uber. By having better data about passenger demand (along with other things, of course), Uber has become the market-changing force we all know.

We all throw ideas around ‘what is a good metric’ and ‘how we can better automate’. This talk helped a lot.


Allison Miller (@selenakyle) closed off the first day with a topic that is very near and dear to me: drawing concepts from Economics into InfoSec and Risk. I’m a huge fan of her work, and this was no exception. Following a quick look into how microeconomics topics such as maximization of utility and utility curves work, she clearly demonstrated how, given an expected value (mean), a posture of risk aversion manifests itself as the desire for smaller expected variance. She then chose to explore possible linkages between InfoSec/Risk and macroeconomics topics, including a great tie-in to the Lucas critique. She has mentioned before the possible use of a ‘Security CPI‘ but now called out the possibility of defining ‘security econometrics’. Very thought-provoking indeed.


Day 2 post coming up soon…


NOTE: If this summary is at all interesting, know that SIRA recorded the event and that, if I understood it right, video will be made available to members (hint, hint, …) soon.

#BSidesSF, day two (kind of)

On this second day of BSidesSF, things were more familiar: figuring out which track was where, what to expect in terms of flow, etc… The acoustics seemed to improve a bit, the crowd was a little thinner – it is Monday, after all… But still lots of interesting people and sessions.

Because of other commitments, I had to leave right around lunch, so only had three sessions to attend. 😦

Still, on to the sessions:

  • In what was, in my opinion, the BEST session at BSidesSF, Tony Martin-Vegue (@tdmv) did a phenomenal job of delivering an informative, clear and entertaining session on the [mis]use of Statistics in information security scenarios. He presented good points about how things such as surveys (always a favorite of vendors…) and charts can be fraught with peril if taken at face value. He discussed the issue of ‘semi attached figures’, when one data point is not proof of another, but are presented together to confuse the subject. Finally, he hinted at other biases and discussed recommended practices (assume good intentions, but be skeptical!) and links to good books on the subject. I’d be remiss if I didn’t mention that not only were the supporting slides hilarious, but his flow of demonstrating some concepts was amazing: his deconstruction of 3D pie charts, or the powerful visual of how misleading a line graph can be. If you watch only one session, make it this one!
  • An interesting concept in statistics is “reversion to the mean”: after an extreme measurement, the next one may be closer to the average. It applied here: after a great session, the next one kind of missed the mark. Originally entitled Ground Zero and meant to discuss trends in banking malware (or so I read), it turned out to be a high-level description of some data that was available from a sensor network, followed by a description of how companies already have “interesting” data lying around and should share it. Jonathan Curtis made a good attempt at engaging the audience and I praise him for having answered quite a few questions, but ultimately the content was not there.
  • My last session was on Phishing, delivered by Kevin Bottomley (@k3v_b0t). He presented an overview of key points about phishing – usage, how it looks, how it flows, how it can be created, … – then demonstrated some of these ideas on a demo system/account. It was impressive to watch how quickly a campaign can be created and launched. Also, it may sound simple but I really liked how he ‘lifted the curtain’ just a tiny bit on how advanced phishing detection leverages Machine Learning algorithms such as Natural Language Processing. If anything, I think it’d be really cool to explore that angle further. Sidenote: during his talk, Kevin had to deal with “demo gremlins” (some things didn’t work as expected), but he handled things really well…

And that was that… I left OpenDNS soon after: there was a draw for some prizes (t-shirts, books, and some electronics) but the odds were not in my favour.

I’m extremely thankful to the organizers, the sponsors (OpenDNS in particular for hosting the event), the presenters and the audience. I truly hope I can make BSidesSF a regular stop in my calendar if I keep coming back to RSA.

#BSidesSF, day one.

Having attended BSides Toronto a couple of times (and always having a great time), I was very excited to attend BSidesSF, just ahead of the RSA Conference. This is a simple post on observations and the sessions I watched during “day one”.

Yes, BSidesSF is two days and two or more tracks, much larger than my reference with BSides Toronto. This means that not only I had to choose which sessions to watch, but that there’s more tomorrow. 🙂

I just noticed the videos for some talks are already at IronGeek. Talk about fast turnaround!

  • Venue: the OpenDNS offices. While acoustics can be a challenge sometimes, the venue itself is great: spacious, easy to access, bright. Someone mentioned it would get warm in there, but frankly I didn’t notice. WiFi worked well (I hope everyone behaved) and it was pretty easy to get from session to session.
    Meals: breakfast and coffee breaks in the back (oh, the bagels and donuts…), food trucks in the alley for lunch. Good food, great conversations.

On to the sessions:

  • Gopal Jayaraman from SierraWare (@SierrawareLLC) started off (early by about 10 minutes, but we were all ready anyway) with a good discussion on how the Certificate Model is broken and it can be abused. He then introduced the topic of Certificate Pinning – the practice of ‘locking’ use of specific certificates within the application itself so as to prevent Man-in-the-Middle attacks. While a good feature for privacy, it makes network traffic opaque to monitoring systems. He then went on to describe the approach his company is taking by offering a “Virtual Mobile Infrastructure” (VMI, similar to VDI) that can address the Certificate Pinning issue – and many other mobile security issues – by providing a secure gateway that can intercept the application calls and network traffic. An eye opener and good map for further readings on my part…
  • Next session I attended was a look into practical countersurveillance tips by Lisa Lorenzin (@llorenzin) – title of the talk was a bit edgy (go look it up…) and the content (and delivery) was solid: a call to action for individuals to evaluate how comfortable they are with surveillance and what they want to do about it. This can go from simple changes to behaviour (use HTTPS, use different search engines, …) all the way to moving services away from the Googles of the world and into your own servers. As she mentioned during the talk, this is not to protect against targeted surveillance, but more as a statement against the bulk collection done nowadays. Lots of references to Snowden and politics, the good work done by the EFF and others.
  • Last session before lunch had a last minute change: Russell Thomas (@MrMeritology) couldn’t make it and Allan Friedman (@allanfriedman) stepped in with minimal advance notice. He did a great job describing the broad strokes of Economics of Information Security (a topic near and dear to me) and described the work he’s involved in: helping the government stimulate voluntary cooperation between multiple stakeholders in issues related to cybersecurity. He touched on what the main topics of interest are – network & infrastructure security, web security & consumer trust, and processes & markets – and called upon us to comment on who are the key partners that should take part in this conversation. Public policy is usually a dry subject, but he did a great job explaining what he’s up to.
  • The afternoon sessions started with Josh Pyorre (@joshpyorre) describing his approach to adding Intrusion Detection to cloud-based systems, with detailed demos on how his IDS – based on Apache and Snort – can be placed in-line to protect traffic bound to cloud-based Web servers. I really liked the detailed technical content on the session, though I think the approach needs to be expanded when dealing with larger sites that take advantage of cloud-based scaling (think AWS’s Elastic Load Balancer). Still, a good session from a solid presenter.
  • I then attended Lucas Zaichkowsky‘s (@LucasErratus) session on state of password security – hint:very poor! – and a simple explanation arond the basic crypto concepts involved – things like salt, extra rounds, etc…. He then described his experience running password cracking on a sample of known passwords out there. With minimal hardware – less than $1000 worth – and a good strategy for directing the computing efforts, he was able to recover a significant number of passwords. He wrapped up with recommendations, particularly a focus on defensive planning, including broader adoption of two-factor authentication and use of password management solutions. Really nice presentation.
  • In “Analyze This!”, Aaron Shelmire (@AShelmire) covered some ideas behind the basics of Data Science as it relates to security, and then delved into examples of ‘features’ that are present in malware than can lead to identifying it among the vastness of network traffic. It was extremely entertaining and informative to review packet captures of malware samples such as Zeus, Upatre and Pirpi, among others, and identify key ‘tells’ that these strains have. He also discussed similar analysis of endpoint analytics (registry keys, services, etc…) and wrapped up with examples on user analysis: how differences in behaviour between users and attackers are apparent when looking at RDP sessions and logon activity. Extremely interesting!
  • Finally, Robert Lucero (@jediguybob) wrapped up the day with a nice discussion on the use of internal certificates for protecting microservices architectures. He showed some nice demos illustrating the concepts and delivered it flawlessly. If nothing else, his talk was a great reminder that implementing certificates without proper care and planning can lead to nasty consequences, as even giants such as Microsoft, Google and Amazon have seen.

In addition to the sessions, I was able to catch up with former colleagues, meet new people and catch up with some folks I’d only known on-line. And there’s more tomorrow, as we dive into Statistics, Malware, and other topics.

Many thanks to all the organizers, speakers, volunteers, and sponsors. Great conference so far!