The Market Research industry is lagging behind AdTech and MarTech… ResTech to the rescue!

The Market Research industry is lagging behind AdTech and MarTech… ResTech to the rescue! — Part 3/3

by Hannu Verkasalo, Swash Advisor

If you haven’t already, read the first and second editions of this series.

Swash is a Web 3 native, blockchain-based, clickstream data company, and is entering the market research industry with a number of unique solutions and a freshly-built panel with behavioural online data collection.

However, what is the market research industry all about, where did it come from, and what opportunities and challenges exist? How could Swash pose changes therein? In the final blog entry of this series, let’s take a look at some of the more fundamental challenges of MR more specifically, and discuss what we are aiming to do within Swash to induce change.

First, traditionally panelists have not been part of the upside in MR. This is quite significant. After all, almost all the relevant data within MR is collected from consumers (of course there is market research focused on non-human activities, too, but most of the research work is somehow based on understanding human activity), from real people.

The data being collected, again, might be online or offline survey data, interviews, or as is most often the case today, be based on data streams coming from passive observation of the activities these panelists are doing on their multitude of devices — TVs, radios, tablets, smartphones, computers, smart speakers etc — or their credit cards or online bank accounts etc.

Some of this data is being collected fully passively (people simply need to opt in and install or set up some HW/SW for the data collection), in some of the MR tasks people need to actively participate — for example to follow a pre-determined or random sets of actions when being observed by the MR company, or fill in questionnaires.

However, the main point is that in most, if not all, of today’s panels, the relevant incentives are fixed, not related to the value or use of the data collected from panelists. That’s right, people need incentives and motivation for them to take part of the surveys.

As mentioned above, there is no transparency, so the intrinsic motivation that could be hypothesised to come from the fact that people see how their participation helps the world or other people, is not there.

So, panelists need to be compensated somehow in return to them giving data to the MR company. Most panel companies, research projects, experiments, and ongoing polls provide some kind of lottery, or as is often the case, provide fixed payments or gifts/vouchers to people who participate in the respective research.

This leads to a dynamic where people start to simply optimise on that, participating in as much research as they can, filling in surveys as quickly as they can (irrespective to the quality of their answers), they become “MR participation machines”, or as the industry increasingly calls them — and considers as bad — “professional panelists”. Naturally this use of fixed incentives also restricts the pool of people who are interested in participating in the first place, in other words, causing significant sample bias.

In stark contrast, Swash puts the upside into the data owner’s (panelists’) hands! In Swash’s approach, nobody is yet putting the dollar value on the data you hand over when the data is being collected. Instead, it is being tokenised — you receive tokens if you share your web clickstream data, for example. These tokens, then, decrease or increase in value, according to demand.

When MR data buyers arrive to purchase and access this valuable data being collected in the Swash ecosystem, they need to put down money (in any format they see fit), and this money gets converted into the tokens that these panelists hold — therefore providing liquidity.

The more demand there is (more active use of this data by one buyer for more use cases, or simply more users and buyers of data arriving), the higher the value of the tokens related to the datasets being available for sale. This leads into a self-enforcing dynamic; panelists feel that they have upside, their incentive is not fixed. They remain in the panels for the long-term (not quitting quickly), they think about the data they share (both quantity and quality) more seriously, they are keen to participate in new/incremental ways of data collection — in other words they are keen to take whatever actions are considered valuable by the demand side of the MR data industry.

Swash, therefore, eliminates middle-men, quantifies and materialises the monetary value of the demand, and channels this directly, with all the upside being included, to the panelists.

The second existing major challenge is transparency and privacy. We need to ensure that panelists understand what is being done with their data. The ones who provide the data (panelists = typically the objects of research in MR) should be considered as stakeholders, the owners of data. They might be different from data collectors (who run operational platforms, like passive behavioural data collection platforms, consumer profiling platforms in AdTech, or survey operations), but they still should own the data collected from them, and have something to say about its use — or at least be legally in charge of deciding what kind of legal or operational agreements or arrangements are done with their data.

Unfortunately, historically they have had very little visibility into anything, and there have been serious breaches of privacy by panel companies, especially publicly listed MR giants, with most big MR operations being black boxes for anybody outside. As a result, it is difficult to understand how they operate and treat panelists, or their data.

For example, if you have been part of any of the big survey panels over the past decade, or if you have used many of the web services or mobile apps over the past couple of decades, it’s likely you have little knowledge on how these companies run or provide these panels, how sites or apps have used your data, and where it ends up. Generically in MR, panelists or people who participate in different types of MR projects or data collection rarely have any insight into the use of the data collected from them.

If you ask a random panelist, participating in today’s online survey panels, what kind of causes their participation has helped, or what kind of insights have been derived from their survey answers, almost all would say they have no clue whatsoever.

Not only is this a negative thing for anybody’s motivation to take part in these studies (causing ex-ante bias in the data collection already in itself), but ethically and juridically speaking, this cuts the owner of data from any visibility to the use of their data. So, what happens after the data collection?

Legally MR companies might have reserved rights to use this data for a variety of abstract reasons, or to create derivative products, or even to sell or license this raw data further to their own clients, but not revealing exactly to whom, what, why and where. Of course GDPR in the EU, and CCPA (California), were built over recent years to address some of these challenges, but most of the MR companies have addressed this by vastly complicating and amending their privacy policies — reducing visibility and causing more haziness, rather than providing more visibility or transparency.

In any case, what I consider important in the future is to give panelists also a chance to understand where their data goes. Or even better, be able to manage how their data is being distributed (in which platforms, or in which form), in which dimensions (demographics, behavioural, opinions, etc.), at which price (one-off data sales, monthly subscriptions, free donations etc.), where (in which countries, through which data exchanges or companies), for which causes (e.g. political polling vs. clickstream consumer journey mapping etc.), and how (for example just when aggregated with other people’s data, not individually).

Consumers, panelists, and data owners, should have more visibility into the use of data, with a full paper trail of what happened, and who can touch their data. At Swash, we build from blockchain-based technology, transparent ledgers, and independent distributed structures that form the basis of this transparency and data flows.

These transparent ledgers, archives of data flows and transactions, simply did not exist earlier — unlike in DeFi or other industries that have embraced blockchain already. Swash is built on top of this ideology. We want to provide controls so panelists can decide what happens with their data, driving transparency in the data economy. This is achieved via our technical infrastructure and design, but also the UI elements in our services.

Moving forward, one of the challenges for the MR industry is regarding data deals and transactions. There have been no centralised places for discovering, validating, accessing, and trading data. MR data deals have been made on a one-off basis, through manual negotiation, or by two or more companies speaking to each other in private settings.

It is manual work, through human-centric sales forces, business development officers, and others, that have been needed in driving deals to sell and buy data. There have been standardised tools and SaaS dashboards that you can access, but these are not tools to access, use, or commercialise data, rather they are simply visualisations. Modularised, automated and standardised, market places or market layers with enough power and acceptance, do not yet exist in MR.

So, for anybody new to MR, such as new startups focused on creating all kinds of data-driven tools or consumer segmentation models based on existing data being collected by panel companies, or anybody who is interested in developing AI-based intelligence (anything that relates to turning raw data into information, knowledge, or actions) automatically to build on top of MR data — it would be beneficial to interact and transact with an ecosystem which is powered by data market places, and data economies, so that they can focus on what they do well…. their IPRs being built on top of the data layer, building applications and useful tools.

In one word, we need automation. In order to innovate and move forward, and maximise the opportunities lying around in deriving something useful from raw or processed data, we need to eliminate humans, one-off deals, and large transaction and negotiation costs.

For rapid transactions and more innovative deal types (e.g. monthly licensing vs. one-off sales), and dynamic programmatic pricing, crypto/blockchain-based financial and commercial infrastructure is needed, with transparent transactions and openness for new ideas and automatic transactions/ledgers. Datasets and anything valuable in terms of IPRs needs to be tokenised, and made available — the value needs to be decided in the market (not in one-off deals), and the existing building blocks need to be reused. Swash is now making that available for the MR industry!

Furthermore, there have technically been no APIs or standardised layers in the MR ecosystem, so it has been impossible to build e.g. application or data aggregation layers on top of standard elements on the lower level. Different companies in, say, passive behavioural digital data collection, have all had their own data formats, data models, methodologies, taxonomies, with these companies rarely interoperating with each other. This makes the buyer’s or user’s life difficult — they need to essentially integrate one dataset at a time, and figure out how to make different datasets comparable.

For most of the MR data circulating around, there are no widely accepted data formats or standardisation efforts taking place, unlike the metrics and objects of trade in digital advertising, as a contrasting example. The other lacking element is aggregation and layerisation, and lack of standardisation or accepted interfaces therein.

In order to do something useful with MR data, one needs to process it, taxonomise it (e.g. adding categories or meta data), group it, aggregate it, do some calculus, and so on. However, each data pipe (read “MR monopolies”) have their own methods on how to do this, with no interoperability or standardisation.

In the same fashion, there are also no standard APIs that would power up the use cases of discovering, validating, and integrating MR data assets for any real use. These APIs — if there were some — would have been really nice over the past decade already, when many new actors in MR (such as hedge funds or ad platforms as clients) would have been interested in tapping into MR data to improve their business… but to no avail.

Swash is now making some of the key elements (data collection and its availability through market places and APIs) available, so that we achieve better modularisation around behavioural digital data, and can induce and welcome applications that work independently on top of Swash data, such as AI-based tools to create timely insights.

This type of layerisation and technical modularisation is needed, so that all companies can focus on their respective strengths. We are providing tooling and APIs, so that anybody case use this data, as long as the panelists will get something. But we love automation, APIs, standardisation and transparency. All this that we are doing, will help the MR industry to achieve more rapid development in the future.

Finally, for any innovation to happen, in addition to automation and data exchanges, and new ways of panelist incentivisation or the standardisation/modularisation and APIs (above), more open-source momentum and an open innovation mindset for both technical and business development are needed in MR. Older companies or people do not change quickly, and old mindsets are even slower to change.

If you are not pushed for competition, and are able to rely on a business position which is monopolistic, there is hardly any upside for new investments to be made, or not worth sweating for new innovation either. However, any lowering of barriers to entry and an open industry structure and technical architecture (which are driven by most of the factors explained above) welcome new companies, new minds, new innovation, new challenger technologies, and call for change.

This also attracts venture capital, all kinds of brilliant people, and provides the baseline for new technologies (even from other industries, like blockchain), to have an impact. More than anything, open source, as it has invoked big changes already in finance, databases, and AdTech, can have superior effects on the slow-moving world of MR.

One of Swash’s aims is to push MR to adopt more open-source ideas. We also want others in MR to realize the potential of open collaboration, realising that this industry can be much bigger if we collaborate, plug and play, build on top of each other, and build total value (not just value for our own private enterprises).

Overall, it is worthwhile to note that the overall MR industry has been growing rapidly, and at times reaching growth rates of 10% per year. This has led to an increasing number of new ventures, with new ideas emerging, despite all the challenges identified above. However, considering the rate of change, and when referring to the comparable growth in other areas close to marketing and advertising (like AdTech and MarTech, achieving 30–100% annual growth at times), market research, and specifically the challenges of getting ResTech (research technologies) off the ground, have yet to realise their potential.

Research, data, and insights are increasingly needed in making the best business decisions, and AI — along with a spectrum of other enablers like big data, cloud computing, programmatic advertising — could lift the overall MR industry to a much more elevated stage, and push future growth. We at Swash are optimistic about this potential being finally realised over the coming decade.

Written by Hannu Verkasalo; Swash Advisor and expert in the field of data analytics, Market Research & big data for 20 years.

About Swash

At Swash, if it’s your data, it’s your income.

Swash is an ecosystem of tools and services that enable people, businesses, and developers to unlock the latent value of data by pooling, securely sharing, and monetising its value.

• People share their data to earn while retaining their privacy.
• Businesses access high-quality, zero-party data in a sustainable and compliant way.
• Developers set up and build systems within a collaborative development framework with ease.

As the world’s leading Data Union, Swash is reimagining data ownership by enabling all actors of the data economy to earn, access, build and collaborate in a liquid digital ecosystem for data.

Join the biggest data movement in Web 3:
Twitter | Telegram | Reddit | LinkedIn | Website | DAO Ignition

Nothing herein should be viewed as legal, tax or financial advice and you are strongly advised to obtain independent legal, tax or financial advice prior to buying, holding or using digital assets, or using any financial or other services and products. There are significant risks associated with digital assets and dealing in such instruments can incur substantial risks, including the risk of financial loss. Digital assets are by their nature highly volatile and you should be aware that the risk of loss in trading, contributing, or holding digital assets can be substantial.

Reminder: Be aware of phishing sites and always make sure you are visiting the official https://swashapp.io website. Swash will never ask you for confidential information such as passwords, private keys, seed phrases, or secret codes. You should store this information privately and securely and report any suspicious activity.