Building a Curated Knowledge Base
Content discovery presents a multifaceted challenge with technical and political complexities. The goal is to distribute content in a manner that is both democratic, ensuring accessibility to a wide audience, and targeted, reaching the most relevant recipients — but finding the balance between wide accessibility and necessary restrictions can be challenging, especially given the potentially severe impact of certain content, which often necessitates government regulation.
Over the years, different digital platforms have tried to address this issue through a set of approaches, with varying degrees of success. The underlying challenge lies in the ‘effective distribution of information,’ which inherently demands a compromise. On one hand, individuals should have the autonomy to determine the content they wish to access, ensuring they can find the most relevant information without encountering manipulation and restrictions. On the other, the information consumed should refrain from infringing upon the freedoms of others. And, it’s essential to manage the harmful content without introducing a ‘single point of failure.’
-
When we examine the initial steps in this direction, Netscape stands out as one of the earliest and most successful web browsers and pioneers in shaping the internet’s evolution with the aim of making information more widely available and accessible to the general public. However, it ultimately failed to maintain its dominant position in the market because it could not effectively bring together users and content providers. And it was eventually phased out as it was slow to adopt the transition to web standards and open-source development that allowed other competitors to gain ground.
Google search solved this problem much more effectively with the Google Auction system. In this system, advertisers place bids on specific keywords that they want their ads to be associated with. When a user conducts a search using one of these keywords, an auction is held to determine which ads will be shown on the search results page. The ads that are shown are those that Google determines to be the most relevant and valuable to the user, based on the bid amount and other factors such as the quality of the ad and the relevance of the landing page to the search query. This was revolutionary because, for the first time, there was a better and — a more effective — way to bring users and providers together without sacrificing from the user experience.
-
In the modern digital landscape, numerous highly successful internet companies, such as Google, Meta, and Twitter, have immense power in navigating the content discovery, and this can be proplematic in so many different levels. They leverage algorithms to heavily manipulate the ranking of web pages in search results (leading to significant bias in discovery), rely on recommendation systems that are based mainly on users’ past interactions and preferences (creating “filter bubbles” and echo chambers), implement content moderation tools that conform to certain, possibly controversial demands related to social and political issues (e.g the censorship/limitations of ChatGPT), and they also use problematic ad targeting to meet the specific needs of the platform (e.g Elon Musk’s weird ad targeting strategy). These practices represents a critical impediment to the genuine exploration and discovery of authentic content.
As former Twitter CEO Jack Dorsey clearly puts in his article; “The problem today is that we have companies that have both **protocol** and **content discovery**.” I agree wholeheartedly that the ideal is; “a ‘follow’ action should always deliver every bit of content from the corresponding account, and the algorithms should be able to comb through everything else through a relevance lens that an individual determines’’. Perhaps as Jack said and prominent mathematician Stephen Wolfram would agree; “instead of a company or government building and controlling these solely, people should be able to build and choose from algorithms that best match their criteria, or not have to use any at all.’’
According to many who support the efforts in this arena, the only known way to truly live up to these criteria is a free and open social media protocol that is not owned by a single company or group of companies and is resistant to corporate and government influence (decentralization). And in this regard, there are many promises to decentralize social media like NOSTR or AT Protocols in which users to choose from algorithms and protocols to access desired content.
However, the major issue with the expectation that people to set up their own algorithms or protocols to access the content they desire is — as Wolfram says — “unrealistic’’. Most people lack the necessary technical knowledge and detailed information to configure such systems themselves. Therefore, Wolfram’s perspective suggests that a more practical approach is to rely on separate third-party providers who can customize the experience for specific user groups. This way, the scope can remain focused without resorting to centralized bans on ‘malicious content,’ thereby curbing the dissemination of harmful information.
Whether we choose to move towards decentralized systems or rely on third parties to manage information, a common consequence is a decrease in the number of users who can access information and a significant amount of restriction on the available content. Until we come up with a solution that strikes a balance between accessibility and control, information will still remain subject to limitations, independent of corporational or other centralized control. So the problem persists.
-
Perhaps there’s another way to empower individuals to choose the content they want, without needing to make these compromises.
Is it possible for a constantly expanding knowledge base to ensure the accessibility of quality content for all users through its technology without inadvertently assuming the role of an internet gatekeeper?
The surge in the creator economy has underscored the value of curated digital content, characterized by the deliberate selection, organization, and presentation of material. This approach is a boon for content consumers as it helps them sift through the vast volume of information, effectively reducing information overload by eliminating irrelevant or subpar content. Additionally, when individuals engage in genuine content curation, it can expand consumers’ horizons by introducing them to new ideas and information, all without requiring centralized control or manipulation in deciding what content is accessible.
A knowledge base curated by individuals with diverse interests, supported by appropriate technology for content dissemination, serves to enrich the user experience by providing easier access to a broader and more comprehensive range of information. This approach ultimately saves time and resources while avoiding the creation of restrictive “filter bubbles”.
Moreover, these systems can support learning and collaboration by enabling users to share their knowledge and expertise.
-
I believe there are numerous ways to access content efficiently, and I think it’s crucial that these methods remain free from political bias and unnecessary complexity. Technology, in my view, is a powerful catalyst in achieving this objective. I am optimistic that we ultimately will cultivate a more efficient and decentralized information ecosystem by fostering increasingly social and collaborative systems in the future.
Thanks for reading it!
