Posts

Posted on

Personal Data as an Asset

There is a well worn axiom in business that ‘data should be treated as a corporate asset’.  This is, of course, very true and the advances in data science and ‘big data’ are giving the potential for that data to become even more valuable.

This got me thinking about how personal data should be thought about in the same way.  Think about all the data generated from what you watch, what you listen to, where you visit, what you review, data from wearables, etc.  All of this data is consumed and analyzed by 3rd parties currently, but what if individuals were able to take control of, what is, after all, their data.

Would this give rise to data science companies marketing algorithms directly to consumers (much like pharmaceutical companies market drugs directly)?  Could it also give rise to the equivalent ‘data quackery’ similar to the natural supplements and homeopathic industry?  That is, junk algorithms that, at their most benign, do no harm and at their worst incent you to dangerous courses of action?

Would there also be a new industry for ‘personal data scientists’ (like financial councilors or tax advisers) that would help you assess all of the data assets you have and how to best combine or leverage them with third parties to your best benefit (and not just the benefit of 3rd parties)?  Wouldn’t it be great to have some control over the hundreds of arbitrage-like transactions that go on behind the scenes when you are waiting for a page to load on a commercial web site via browser setting that allow you to control what information about you gets shared (and with companies).

Posted on

Teaching AI To Be ‘Smarter’ By Doubting Itself

Interesting post that suggests that in deep learning algorithms, questioning things may lead to higher quality conclusions.

Researchers at Uber and Google are working on modifications to the two most popular deep-learning frameworks that will enable them to handle probability. This will provide a way for the smartest AI programs to measure their confidence in a prediction or a decision—essentially, to know when they should doubt themselves.

Deep learning, which involves feeding example data to a large and powerful neural network, has been an enormous success over the past few years, enabling machines to recognize objects in images or transcribe speech almost perfectly. But it requires lots of training data and computing power, and it can be surprisingly brittle.

Somewhat counterintuitively, this self-doubt offers one fix. The new approach could be useful in critical scenarios involving self-driving cars and other autonomous machines.

“You would like a system that gives you a measure of how certain it is,” says Dustin Tran, who is working on this problem at Google. “If a self-driving car doesn’t know its level of uncertainty, it can make a fatal error, and that can be catastrophic.”

Posted on

What Does An Enterprise Architect Do?

‘Enterprise Architect’ is a very fashionable title these days which causes a bit of confusion (and consternation) for actual EA practitioners. Typically, this title is attached to the role of someone who has deep technical knowledge about a given technology/application/suite. This is not an Enterprise Architect.

This article does a great job of clarifying what Enterprise Architecture is and what an Architect does (or should do).

“Before answering that question, it is important to note that no architecture is a solution. Often people confuse a solution, such as corporate infrastructure, as the architecture. This is an all too common mistake and very misleading. An architecture guided the development of the infrastructure, the infrastructure is a solution – not the architecture.

“The architect’s role isn’t to create solutions. Rather the architect’s role is to inform decision-makers and guide development of solutions based on understanding business drivers and needs of the organization. Although a single person can have both a role as an architect and a developer. The architect typically takes a broader and material independent view than the developer, yet leaves much downstream detail to the development community.

“So, since architecture is not a solution what is it? It is a package of information that describes what is needed to achieve a given result and what it might look like in a future state if implemented. In order for an architecture to be effective, that is for it to be realized in solutions, it must guide decisions.

“Any good architecture addresses a well-defined scope and seeks to achieve specified goals. For example, an architecture for a back-office software suite will seek to enable improvements to back office operations, an architecture for a department network will enable department interconnectivity, an architecture for corporate infrastructure will address needed services throughout at lower costs, etc. For each scope there are decision-makers that can either accept or reject the guidance from the architect such as office managers, network managers, the head of IT, etc.

“Those that deal with Enterprise Architecture take the broadest view, deal with issues that are oftentimes beyond even the corporate level, and are most effective when they influence corporate or Board level decision-makers.

Posted on

Unpredictions for Artificial Intelligence (AI)

This post is a refreshing counterpoint to the breathless ‘AI will take over everything’ reporting that is increasingly common of late.

Self-driving cars
The first area is that “we won’t be riding in self-driving cars”. As Dr. Reddy explains: “While many are predicting a driverless future, we’re a long ‘road’ away from autonomous vehicles.” This is is terms of cars that will take commuters to work, a situation where the commuter can sit back and read his or her iPad while paying little attention to the traffic outside.
He adds: “For a number of years ahead, human operators and oversight will still rule the roads, because the discrete human judgments that are essential while driving will still require a person with all of his or her faculties — and the attendant liability for when mistakes happen. Besides technical challenges, humans tend to be more forgiving about mistakes made by human intelligence as opposed to those made by artificial intelligence.”
Disappearing jobs
The second ‘unprediction’ is that people will not be replaced by AI bots this year. Dr. Reddy states: “While it is possible that artificial intelligence agents might replace (but more likely supplement) certain administrative tasks, the reality is that worker displacement by AI is over-hyped and unlikely.” So robots won’t be taking over most jobs any time soon.
This is because, the analyst states: “Even in an environment where Automated Machine Learning is helping machines to build machines through deep learning, the really complex aspects of jobs will not be replaced. Thus, while AI will help automate various tasks that mostly we don’t want to do anyway, we’ll still need the human knowledge workers for thinking, judgment and creativity. But, routine tasks beware: AI is coming for you!”
Medical diagnosis
The third aspect is that we won’t get AI-powered medical diagnoses. This is, Dr. Reddy says “Due to a lack of training data and continued challenges around learning diagnosis and prognosis decision-making through identifying patterns, AI algorithms are not very good at medical decision automation and will only be used on a limited basis to support but not replace diagnosis and treatment recommendations by humans.”
He adds: “AI will be increasingly deployed against sporadic research needs in the medical arena, but, as with fraud detection, pattern recognition by machines only goes so far, and human insight, ingenuity and judgment come into play. People are still better than machines at learning patterns and developing intuition about new approaches.”
Importantly: “People are still better than machines at learning patterns and developing intuition about new approaches.”
AI at work
The fourth and final area is that we will still struggle with determining where artificial intelligence should be deployed. Dr. Reddy states: “Despite what you might be hearing from AI solution vendors, businesses that want to adopt AI must first conduct a careful needs assessment. As part of this process, companies also must gain a realistic view of what benefits are being sought and how AI can be strategically deployed for maximum benefit.”
The analyst adds: “IT management, business users and developers should avoid being overly ambitious and carefully assess the infrastructure and data required to drive value from AI. Best practices and ‘buy versus build’ analysis also should be part of the conversations about implementing AI applications.”

Posted on

What Is Missing From Big Data

This is an excellent TEDTalk on what is missing from bigdata (hint: it is the human element).

Why do so many companies make bad decisions, even with access to unprecedented amounts of data? With stories from Nokia to Netflix to the oracles of ancient Greece, Tricia Wang demystifies big data and identifies its pitfalls, suggesting that we focus instead on “thick data” — precious, unquantifiable insights from actual people — to make the right business decisions and thrive in the unknown.

An interesting (but not too surprising) stat from the intro is that 73% of all bigdata projects deliver no value.

Posted on

Turn Off All Your Push Notifications

Really? Someone had to write a 500 word ‘article‘ about what should be common sense?

There’s a solution, though: Kill your notifications. Yes, really. Turn them all off. (You can leave on phone calls and text messages, if you must, but nothing else.) You’ll discover that you don’t miss the stream of cards filling your lockscreen, because they never existed for your benefit. They’re for brands and developers, methods by which thirsty growth hackers can grab your attention anytime they want. Allowing an app to send you push notifications is like allowing a store clerk to grab you by the ear and drag you into their store. You’re letting someone insert a commercial into your life anytime they want. Time to turn it off.

Posted on

Welcome to Our Startup Where Everyone is 23 Years Old Because We Believe Old People Are Visually Displeasing and Out of Ideas 

https://www.mcsweeneys.net/articles/welcome-to-our-startup-where-everyone-is-23-years-old-because-we-believe-old-people-are-visually-displeasing-and-out-of-ideas

This is hilarious because it is true. I’ve seen so many ‘startups’ spend a huge amount of money and effort trying to imitate the trappings of a startup rather than having original ideas and actually producing something. Here is a sample (more at the link above):

Hello, and welcome to our startup. We hope you’re enjoying your complimentary snifter of vaporized coconut water. Once you’re done, please place the glass into one of the blue receptacles around the office, which will send the glass to be washed and dried. Do not place it into one of the red receptacles. The red receptacles take whatever you put inside of them and launch it into space.

If you look to your left, you’ll see one of our employees using a state-of-the-art ergonomic sleeping desk. Most startups have standing desks, but we have sleeping desks, dancing desks, and even skateboarding desks. The skateboarding desks are just large skateboards you can use to skate around the office. Be careful while skating, though, because we don’t offer any sort of medical insurance, since our benefits budget all goes toward cool desks.

Posted on

Volvo’s Electric Car Announcement

I think that Volvo’s announcement regarding electric vehicles (EVs) has been largely misunderstood or mis-reported as them stating that they will only have EVs by 2020. Actually, they stated that they will only design and release NEW EVs after that date. The existing stable of gasoline powered vehicles will continue to live on past 2020.

Posted on

Headphones Evesdropping On You

This is an older story, but it has come around again recently.

Researchers at Israel’s Ben Gurion University have created a piece of proof-of-concept code they call “Speake(a)r,” designed to demonstrate how determined hackers could find a way to surreptitiously hijack a computer to record audio even when the device’s microphones have been entirely removed or disabled. The experimental malware instead repurposes the speakers in earbuds or headphones to use them as microphones, converting the vibrations in air into electromagnetic signals to clearly capture audio from across a room.

But, as it turns out, this is less of an out-and-out hack, but just simply taking advantage of a somewhat questionable ‘feature’:

But the Ben Gurion researchers took that hack a step further. Their malware uses a little-known feature of RealTek audio codec chips to silently “retask” the computer’s output channel as an input channel, allowing the malware to record audio even when the headphones remain connected into an output-only jack and don’t even have a microphone channel on their plug. The researchers say the RealTek chips are so common that the attack works on practically any desktop computer, whether it runs Windows or MacOS, and most laptops, too. RealTek didn’t immediately respond to WIRED’s request for comment on the Ben Gurion researchers’ work. “This is the real vulnerability,” says Guri. “It’s what makes almost every computer today vulnerable to this type of attack.”

Posted on

Scalable Microservices

Here is an excellent, in-depth discussion on creating scalable microservices.

In this article, we will look at microservices, not as a tool to scale the organization, development and release process (even though it’s one of the main reasons for adopting microservices), but from an architecture and design perspective, and put it in its true context: distributed systems. In particular, we will discuss how to leverage Events-first Domain Driven Design and Reactive principles to build scalable microservices, working our way through the evolution of a scalable microservices-based system.

Posted on

Reactions To Failure

Great article on system failures in IT and how groups/people react to them. Here is a summary:

tl;dr: Catastrophic system failures are remarkably common in IT-dependent environments. The reactions to such failures varies but is often some version of blame-and-train. There are a number of problems with blame-and-train but perhaps the most important is it is a form of organizational blindness that forestalls improvement.

Three things:

  1. These failures are markers of systemic brittleness, the inverse of resilience.
  2. The blame-and-train reaction is a diversion, a red herring, and counterproductive; it increases brittleness.
  3. There are productive reactions to failure but they are difficult to accomplish, especially when the failure has big consequences.
Posted on

Serverless Approach To Testing

Here is a thought provoking article on testing in a serverless environment:

Serverless architecture uses a lot of services — hence why some prefer to call the architecture “service-full” instead of serverless. Those services are essentially elements of an application that are independent of your testing regime.

An external element.

A good external service will be tested for you. And that’s really important. Because you shouldn’t have to test the service itself. You only really need to test the effect of your interaction with it.

Here’s an example …

Let’s say you have a Function as a Service (e.g. Lambda function) and you utilise a database service (e.g. DynamoDB). You’ll want to test the interaction with the database service from the function to ensure your data is saved/read correctly, and that your function can deal with the responses from the service.

Now, the above scenario is relatively easy because you can utilise DynamoDB from your local machine, and run unit tests to check the values stored in the database. But have you spotted something with this scenario? It’s not the live service — it’s a copy of it. But the API is the same. So, as long as the API doesn’t change we’re ok, right?

To be honest, I’ve reached a point where I’m realising that if we use an AWS service, the likelihood is that AWS have done a much better job of testing it than I have. So we mock the majority of our interactions with AWS (and other) services in unit tests. This makes it relatively simple to develop a function of logic and unit test it — with mocks for services required.