• Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to footer
information matters logo data driven innovation uk

Information Matters

Data-Driven Innovation in the UK

  • About
  • Contact
  • Sections
    • Analysis
    • Funding
    • Interviews
    • News Update
    • Events
    • Policy
    • Research

8 factors shaping the future of big data, machine learning and AI

May 27, 2020 by Martin De Saulles

big data machine learning artificial intelligence

AI and machine learning combined with ever-increasing amounts of data are changing our commercial and social landscapes. A number of themes and issues are emerging within these sectors that CIOs need to be aware of.

At O’Reilly’s 2019 Strata Data Conference in London I got a much better idea where the world of big data, machine learning (ML) and AI may be heading. These sectors have developed rapidly over the last 5 years with new technologies, processes and applications changing the way organisations are managing their data.

The Strata conference provides a good barometer of what the state-of-the-art is in big data manipulation as well as the concerns of developers and users. Eight key points emerged for me from the event.

1. 5G will stimulate the growth of ML and result in new applications and services

I spoke with O’Reilly’s Chief Data Scientist and Strata organiser, Ben Lorica about this and he sees the increased bandwidth and flexibility of 5G as well as the move to edge computing as key enablers. He pointed out that China is a leading global force in this technology but that many firms are still working out the business models for all the 5G investments they are making.

2. Changing skillsets for data scientists

Cassie Kozyrkov, Google Cloud’s chief decision scientist, pointed out in her talk that as the UX for ML tools is improved, the skills required will become less technical and more focused on the ability of data scientists to work across silos and be more integrated into the business.

3. The online and offline worlds are merging

China’s Alibaba ecommerce group and Amazon are experimenting with physical store spaces while bricks and mortar stores are still adapting to the new online world. It feels to me that the offline moves by ecommerce groups are offensive while the online investments by physical retailers are defensive. There is still a long way to go before this fully plays out but the expertise that companies like Amazon and Alibaba have with managing data at scale gives them a key advantage.

4. Internal data platforms are becoming essential for growth and innovation

Presentations from data scientists at Lyft and BMW showed how putting data platforms at the centre of new product development and business process management are driving innovation. While this may come naturally for digitally native companies like Lyft it is also something that traditional, industrial companies are having to engage with as data generating sensors become embedded within products.

5. Open data needs to be taken as seriously as open source software

We all know that open source software is behind the rise of many big data and ML products and services. The commercial and technical case for open source was proven years ago. However, much less attention has been paid to the importance of open data for innovation. The outputs of algorithms are only as good as the quality of the data that goes into them.

Chris Taggart, co-founder and CEO OpenCorporates, the biggest open database of companies in the world, highlighted the problems that companies run into when they rely on proprietary datasets where data provenance may be sketchy and meta data not shared across products. Open data is more transparent and does not lock firms into expensive commercial contracts that can be very difficult for companies to wean themselves off.

6. Importance of capturing and managing real-time data

While real-time or near real-time data is not always required for AI and ML projects, the ability to build systems that can handle it can be a valuable form of competitive advantage. As data-driven decision making become more embedded within organisations the competitive edge will sometimes go to those that can respond more quickly to events. The scale and breadth of offerings from Amazon Web Services in this respect show how the tools to do this are becoming easier and cheaper to access.

7. Legal and ethical issues are starting to change how firms innovate

A talk by Dr Sandra Wachter of Oxford University highlighted an issue that, I suspect, will become more discussed over the coming year or two. She pointed out that many firms are now aware of their obligations to protect personal data as initiatives such as the GDPR have come into force. However, a less discussed issue and one that regulators are still grappling with is that of inference and the decisions that are being made by embedded algorithms based on the data they are processing.

We have a right, in Europe at least, to see what data is being held on us and, to varying degrees, have it corrected or removed. However, we do not have the same redress with the assumptions that firms may be automatically making about us because of this data in areas such as credit checking and health insurance.

8. “To those that have shall be given”

As the conference came to a close, I started thinking about how smaller companies without access to the massive datasets of the internet giants or global FMCG firms will be able to compete in the age of big data and algorithmic-decision making. There is the danger, and perhaps we are already seeing it, of virtuous circles of innovation utilising network effects from online services cementing the position of large companies.

However, as Shivnath Babu, co-founder and CTO of Unravel Data Systems, pointed out to me, the internet and app economy is still capable of allowing small firms to leverage data from their apps and online activities and make an impact on markets. Perhaps this and the rise of open data emanating from public data sources will provide the basis for a new generation of start-ups to change the world in the way that Google, Facebook and Amazon have over the last 20 years.

This story, “8 factors shaping the future of big data, machine learning and AI”  by Martin De Saulles was originally published at https://www.idginsiderpro.com. Reprinted with permission. © IDG Communications, Inc., [2020]. All rights reserved.

 

martin de saulles

Martin De Saulles

Editor, Information Matters

Dr Martin De Saulles is a writer, analyst and lecturer specializing in the commercial applications of data.

  • Twitter
  • LinkedIn

Share this:

  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)
  • Click to share on Reddit (Opens in new window)

Related

Filed Under: Analysis, Features

Primary Sidebar

News Updates

What is the state of European fintech?

January 22, 2021 By Information Matters

Tech.eu in association with Finstar has released a new report, The State of Fintech in Europe. Tech-eu will also be running an event on 10 February to discuss updated findings Read more

UK’s most influential IT leaders

January 21, 2021 By Information Matters

Computer Weekly has published its list of the 100 most influential IT leaders in the UK for the coming year. Based on a list drawn up by their panel of Read more

Building mobile apps? Read this.

January 20, 2021 By Information Matters

Anyone designing, building or managing mobile apps will want to read the latest research report from App Annie, The State of Mobile 2021. Packed with data from their app monitoring Read more

ICO’s Regulatory Sandbox is open for business

January 20, 2021 By Information Matters

Legal experts RPC report that the Information Commissioner’s Office (ICO) is encouraging organisations to make use of its Regulatory Sandbox to experiment with new ways of managing data. The areas Read more

proSapient raises $10m in Series A investment round

January 19, 2021 By Information Matters

proSapient’s SaaS platform for managing and leveraging expert networks in the financial and consultancy fields moved to the next level yesterday by raising $10m in a Series A round. Led Read more

Upcoming Events

Jan 26
9:00 am - 1:00 pm

Ad tech in the UK – data use, tackling harm, and latest regulatory and market developments

Jan 26
2:00 pm - 3:00 pm

Anatomy of real-time at Paddypower Betfair

Jan 27
5:00 pm - 6:00 pm

Gartner: Top 2021 Blockchain Trends

Jan 27
6:00 pm - 7:00 pm

Startup fundraising in 2021: Mattias Ljungman & Alex Wilhelm

Jan 28
January 28 - January 29

Deep Learning 2.0 Virtual Summit

Feb 2
5:00 pm - 6:00 pm

The Decentralized Enterprise and the Future of Work – Webinar

Feb 4
2:00 pm - 3:00 pm

Graph Embeddings for Graph-Native Machine Learning

Feb 11
9:00 am - 1:00 pm

Next steps for location data in the UK – the Geospatial Strategy, market development, regulation, and priorities for research and skills

View more

Footer

Sections

  • Analysis
  • Funding
  • Interviews
  • News Update
  • Events
  • Policy
  • Research

Company

  • About Information Matters
  • Contact
  • Our Privacy Policy

Newsletter


Copyright © 2021 · Information Matters