Telruptive

The economist is featuring a nice article about a new communication medium – transmitting data via light or LiFi. Additionally there is a very nice video on TED about DIDO:

In the future 10Gbps for several meters would be possible. Among some uses is WiFi-like access but via light instead of radio in planes to avoid cabling and interceptions.

The technology seems to be invented, now it is time to find good usage for it. Most new technologies can only be used to their full potential when combined with others.

So what would be possible if you combined:

LiFi with Augmented Reality

At the moment augmented reality means a lot of image recognition in real-time which requires expensive equipment or high-speed backoffice connections. However what would happen if you could combine LiFi sensors with augmented reality. Imagine a data center in which when pointing your phone towards the servers, the LiFi sensor would start receiving real-time information about the server. This would avoid the current drawbacks of sending information about this server over the Internet to then return to the phone. All what would be needed would be a LiFi sensor and a mobile app. No backoffice processing. No security problem with data that has to leave the building.

LiFi with Transport

Your car can negotiate with the objects around it. Traffic jams could be announced from car to car and drivers could be alerted about the fact that they are going to run into a very long traffic jam and should better exit in the next exit.

LiFi with Dating

Put a bachelor LiFi batch around your neck and by pointing your phone towards a desired partner you would get a compatibility score and if the score is high enough some clues about what he or she would like to talk about.

Many more things

There are probably many more possibilities so do not be shy to share them so they can be added to the post…

The big names in dotcom world are busy open sourcing some of their secret sause. It is very important to become familiar with these often strangely named projects because they are responsible for several competitive advantages. Since the list is growing please suggest new solutions in the comments section so they can be added.

Google

• Google has a very long list of open source projects. Some of the most famous projects are Android, Chromium, GWT, etc.  
• Bigtable which by itself is not open sourced inspired the building of Hbase. The same happened for GFS which inspired Hadoop’s HDFS.

Facebook

• Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store
• Hive a data warehouse infrastructure that provides data summarization and ad hoc querying.
• FlashCache is a general purpose writeback block cache for Linux. It was developed as a loadable Linux kernel module, using the Device Mapper and sits below the filesystem.
• HipHop for PHP transforms PHP source code into highly optimized C++. HipHop offers large performance gains and was developed over the past two years.
• Open Compute Project an open hardware project aims to accelerate data center and server innovation while increasing computing efficiency through collaboration on relevant best practices and technical specifications.
• Scribe is a scalable service for aggregating log data streamed in real time from a large number of servers.
• Thrift provides a framework for scalable cross-language services development in C++, Java, Python, PHP, and Ruby.
• Tornado is a relatively simple, non-blocking web server framework written in Python. It is designed to handle thousands of simultaneous connections, making it ideal for real-time Web services.
• codemod assists with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention.
• Facebook Animation is a JavaScript library for creating customizable animations using DOM and CSS manipulation.
• Online Schema Change for MySQL lets you alter large database tables without taking your cluster offline.
• Phabricator is a collection of web applications which make it easier to write, review, and share source code. It is currently available as an early release and is used by hundreds of Facebook engineers every day.
• PHPEmbed makes embedding PHP truly simple for all of our developers (and indeed the world) we developed this PHPEmbed library which is just a more accessible and simplified API built on top of the PHP SAPI.
• phpsh provides an interactive shell for PHP that features readline history, tab completion, and quick access to documentation. It is ironically written mostly in Python.
• Three20 is an Objective-C library for iPhone developers which provides many UI elements and data helpers behind our iPhone application.
• XHP is a PHP extension which augments the syntax of the language such that XML document fragments become valid expressions.
• XHProf is a function-level hierarchical profiler for PHP with a simple HTML-based navigational interface.

Twitter

Twitter open sourced some complete projects (e.g. FlockDB) but especially adds extensions to existing projects. For a full list see here.

Yahoo

• Apache Traffic Server is fast, scalable and extensible HTTP/1.1 compliant caching proxy server.
• Hadoop THE nosql solution at the moment was started by Yahoo. Yahoo actively contributes also to several extensions like Avro and Pig.
• YUI is a free, open source JavaScript and CSS framework for building richly interactive web applications.

LinkedIn

• Azkaban is simple batch scheduler for constructing and running Hadoop jobs or other offline processes
• Bobo is a Faceted Search implementation written purely in Java, an extension of Apache Lucene
• Cleo is a flexible, partial, out-of-order and real-time typeahead search.
• Datafu is Hadoop library for large-scale data processing.
• Decomposer is for massive matrix decompositions
• Glu is a deployment automation platform
• A set of useful gradle plugins
• Indexing engine for IndexTank and API, BackOffice, Storefront, and Nebulizer for IndexTank  
• Kafka is a distributed publish/subscribe messaging system
• Kamikaze is a utility package for performing operations on compressed arrays of sorted integers
• Krati is a simple persistent data store with very low latency and high throughput
• Base utilities shared by all linkedin open source projects
• A set of utility classes and wrappers around ZooKeeper
• Norbert is a library that provides easy cluster management and workload distribution
• Sensei is a distributed, elastic, realtime, searchable database
• Voldemort is a distributed key-value storage system
• Zoie is a real-time search and indexing system built on Apache Lucene

At the moment of writing, Mega Upload was closed by the FBI. Mega Upload represents different things to different people. For Hollywood it represents every form of evil. For others it represents the example of how an industry is alienated from its customers base.

The five why’s

Why are people downloading illegal content that is copyright protected? They don’t want to pay for DVDs and Blu-ray.

Why don’t people want to pay for DVDs and Blu-ray? They are too expensive and not useable in modern households.

Why are DVDs and Blu-ray too expensive and not useable in modern households? €10-20 for a CD, book, DVD or Blu-ray is too expensive if you can get it “for free”. CDs, books, DVDs and Blu-ray do not work on a tablet, mobile phone, smart TV, game box, on-the-go, etc.

How much would people be willing to pay? How would people want to access content? People are likely not willing to pay more than €1 / movie/book/song or even less. Ideally there are all you can eat solutions for €10-20/month. People want to have access in digital format and streaming format so they can take it via their tablet or mobile phone on the go but also watch it from their smart TV without downloading previously.

Why is Hollywood not offering low cost content, all you can eat solutions and digital copies and streaming services globally? Good question. They probably prefer to stick to their old business model and use government regulations to alienate their customers base even more. Hollywood is also scared of distributing content digitally because of fears of canabilizing business and enabling more free sharing.

How can things be done differently?

Assumption: In general people will want to pay for individual content as long as they really want it, the price is low and it is easier to get legal content then illegal content.

Who can charge customers euro cents per transaction? telecom operators.

Who can see if content is distributed illegally? telecom operators.

Who can help users quickly get to legal content? telecom operators.

How can the telecom operators offer a solution that is different from iTunes or Amazon solutions?

Telecom operators have a secret weapon they are not using. Telecom operators know which people are downloading illegal content if they wanted to.

What if your telecom operator would offer a simple set of APIs for content providers to embed in websites to sell individual content at prices between €0.01 and €1, to watermark content and to give content providers a revenue share of 90%?

What if your telecom operator would offer a service for all you can eat content in which for €10-20/month you would be able to access all content (download and streaming) and 80% of the revenue would be distributed according to usage between the content providers?

What if a user signed an agreement that for the individual price or all-you-can-eat price they get the right to use the content for their household and its devices but if they distributed the content they would automatically agree to a distribution license of €1000?

What if all the content that the telecom operator distributes is automatically protected? Proxies would scan all P2P and Mega Upload alternatives to see if watermarks can be detected. If they are detected then the user that purchased the original content is automatically charged the distribution license fee. The watermarked content download would be blocked or substituted by publicitiy for the official alternative. The P2P proxies would cache content hence users that use P2P for non-copyright-protected material would see the performance go up.

How can Hollywood survive on a €0.01-1 / content fee or an all-you-can-eat €10-20 / month? If it is easier to aquire legal alternatives then the number of people that will acquire them will multiply with 10 to a 100. This should mean that overall revenue is likely to be the same only the number of transactions multiply.

What is the advantage for telecom operators? Downloads of content happen any how. The difference is that they will become new revenue stream instead of cost centers.

The system should be offered global and in the next months. Not in 3 years and only in the US and the UK. The window of opportunity is closing and no police force or politician is able to defend a business model that is not aimed towards fulfilling its customers needs. The alternatives will come from crowd funded content generation and over-the-top content distribution, so at the end both Hollywood and Telecom will loose.

Why can Facebook, Google, Salesforce and Twitter role out new features every day and regular telecom operators only every 6 months? Although they are dotcoms, they have thousands of employees and a lot of legacy systems as well. However they are able to roll out a new feature every day, if not every hour or minute and large new systems every so many months, weeks or even days.

How do they do it and how can the telecom industry learn from it?

On highscalability, you will find a lot of information on the architectures of large dotcoms. However if you look at different articles you see that each of the larger dotcoms has an architecture that is shared among different products and services, e.g. scaling messages at facebook.

This is the secret sause of the dotcoms. They have built and continuously improved a highly distributed architecture that can handle millions of users and peta bytes of information. On top of this “shared architecture” go the services. New employees are able to quickly create new services because they do not have to worry about scaling data, monitoring the service, deploying/upgrading versions, backing up data, versioning code, etc.

On the other hands operators have no standardized shared architecture. Instead there is a puzzle of different solutions that often use totally different technologies, hardware, etc. Maintenance and upgrades are a nightmare.

Trying to launch any new service requires a massive amount of planning, lots of different skills, expensive investments in third-party licenses and hardware, etc.

How can you do it differently?

Building a private cloud with virtual servers and storage will not resolve operator’s problems. Just virtualizing the puzzle of solutions is not going to do away with complex integrations.

Operators need to make a more bolt move. They need to separate the new from the old. Legacy systems should be kept and isolated. However a new architecture should be built that works in parallel with the legacy systems. This new architecture should focus on launching new services and partner services at dotcom speeds. Everything should be handled as an independent service. Each service should get its own API. A storage services, a billing service, a monitoring service, a provisioning service, an identity service, a datawarehouse service, a deployment service, a mobile shop service, an inventory service, a support service, etc.

All APIs should use a common technology. APIs for third-parties could use REST. APIs for internal high-load usage could use Thrift or Protobuffers. Each API should have two versions, the easy and the low-level version. The easy API offers the most used but in general basic functionality, e.g. sendSMS(from, to, message). The low-level API offers a complete feature set, e.g. sendBinarySMS, sendSMSWithDeliveryConfirmation, etc. This will allow most services to use the easy API but to have access to the advanced functionality when needed.

Loadbalancing when using the services is key. The loadbalancer is the secret for many rolling upgrades in the dotcom world. An application that uses a certain service will use client-based loadbalancing. By having the loadbalancing be able to receive events, it is possible to dynamically add/remove instances of an API, gradually move requests to a new version of the API, etc.

New service developers will now have to focus on building the business logic for the new service and not on data migrations, scaling, monitoring, backups, etc. The service can have completely new ways of billing and charging, a complex deployment workflow, advanced monitoring requirements, large data storage requirements, etc. However it is not the billing or charging system that has to be extended. Neither a centralized EAI. Nor the monitoring system. Instead it is the service that decides what is best for the service via the use of the easy or low-level APIs. By moving the peculiarities of every service into the service and not into generic OSS and BSS systems, these support systems can be drastically simplified.

Operators should try to focus on launching a lot more niche services and opening up their infrastructure to a long-tail of service suppliers. Instead of general services like PBX for SME, operators should think about hotel reservation services, doctor scheduling services, etc. The value of the operator should be in offering a reliable back-office architecture, assuring service quality and managing the support eco-system. The long-tail of service suppliers should be put to work to launch competing niche offerings and let customers decide which one will survive or not.

Universities are starting to explore the future of the cloud. This future starts by getting rid of the many layers that separate software from physical hardware or bare metal. Currently you need a hypervisor (e.g. Xen, KVM, VMWare), an operating system (e.g. Linux, Windows, Mac OS), a language virtual machine (e.g. JVM), an application server (e.g. Tomcat, JBoss, etc.) and then the application.

In this article, researchers and academics are arguing that there is too much abstractions going on that could be removed in benefit of unseen performance. Projects like Open Mirage, Exokernel and Apache Mesos are examples.

If telecom operators want to offer IaaS and PaaS then they should focus on having a competitive edge that is not currently offered by established providers like Amazon and Rackspace. This competitive edge could be to build a new Cloud OS that has storage and processing nodes that run as close as possible to the bare metal. Building data storage solutions like Hadoop or Cassandra close to bare metal hardware and using the latest solid state drives would offer unseen performance. The cost per user would be substantially lower then less optimized set-ups. Ideally PaaS platforms can be delivered that allow “cloud application servers” to run on base metal. The model would be Heroku on bare metal instead of on Xen+Linux+JVM+App Server+Java App.

Social Enterprise Apps are the next buzz. Companies like Salesforce with Chatter, Yammer, Jive, Google with Google+, etc. all want to change the way employees work in 2012 by adopting Facebook and Twitter-like solutions.

At the moment it is too early to tell who will be the winner. Most products however are still just offering only basic features like status messages, connect to colleagues, share documents, etc.

The real interesting features are still to come. Employee driven process creation and management should make it possible for plain humans (not über-programmers) to define and manage company processes and to transfer a world of Excel, Access and other homegrown solutions to the Web and mobile world.

Operators should jump on the social enterprise apps bandwagon because calls and SMS can still be incorporated into this new portfolio of products. However not in the traditional manner. Since everybody has access to a phone, it could be used for quick approvals either by calling in, getting called or sending an SMS. Even faxes could be incorporated. Traditional companies might be more willing to move from paper faxes to online faxes instead of moving from zero to Facebook speed right away.

The key will be the ability to people to define and manage things themselves without needing support from IT or five level of approvals…

Innovation used to be something related to an R&D department that would experiment with new technologies and a marketing department or product management that would ask customers what new features they required. The business team would be killing any innovation that did not present a business case which complied with company rules: e.g. x% margin, €yM revenue in two years, etc.

Why is traditional innovation no longer good enough?

The cost of launching a disruptive innovation that changes a complete industry has come down dramatically. There are many examples: Skype and roaming, Amazon’s Kindle and paper books, P2P and network bandwidth / media revenue, Salesforce and shrink-wrapped software, iPad and Windows PC, iPhone and Nokia, etc.

Disruptive innovations are more frequent than ever and enablers like Cloud Computing, Open Source, Off-shoring, 3D Printing, etc. allow innovators to launch big solutions on a modest budget.

Most traditional innovation is about evolving a current product by adding new features and improving current functionality. Traditional innovation focuses on prototyping new features and products and showing them to potential customers. However process innovations (e.g. Toyota Production System), business innovations (e.g. freemium), marketing innovations (e.g. Intel Inside), disruptive innovation, etc. are often overlooked.

Every one should innovate

More and more companies are convinced that every one in the organization should innovate and not only R&D and product marketing. By putting special innovation processes in place in which employees can share innovation ideas and use collective knowledge to improve them and get funding, innovation becomes more democratic and often more successful. Companies like Google allow employees to focus one day a week on innovation that can be totally unrelated to their day jobs. People vote with their time which project is worth it. Ideas are shared hence collectively the services get better.

Also upper management is no longer looking from above but should innovate by example. Name all big innovative companies and you see that founders are a big part of innovation and participate in it every week: Google (Larry Page and Sergey Brin), Amazon (Jeff Bezos), Apple (former Steve Jobs), Facebook (Mark Zuckerberg), Salesforce (Marc Benoiff), etc.

Daily Innovations instead of Product Releases

The large dotcoms (Google, Facebook, Amazon, etc.) no longer do market research in the traditional way to find out if users like a feature or not. They also no longer focus on major product releases. Instead they focus on incremental innovations on a daily basis. Users request new features via social CRMs and the most voted features get implemented. Often a feature can have multiple implementations. Users are divided into different groups and new features get enabled for subgroups. If a new feature has a positive effect then it survives and gets rolled out to the rest, if not it gets killed or adjusted.

New products no longer get productized from an idea and afterwards customers are searched for it. Instead customer’s pain points result in paper prototypes that get validated and redrawn until they solve the problem. Afterwards real prototypes are made that get launched in beta or even alpha shape towards real users. Beta can already mean that users are paying for it.

Discovering New Innovations

Discovering new innovations is done by combining groups of people with different expertise (marketing, psychology, arts, technical, business, etc.), to understand a new domain and to question a status quo. Most of the time the best innovations are those that remove a status quo and make a painful activity into a joyful activity, e.g. LinkedIn: networking with people and keeping up to date your business network.

After questioning experts and novice users, innovative companies also observe how users use their products. Often heavy users or first-time users are unsatisfied with current products. New ideas are shared inside but also outside of the company with a network of experts as well as people with completely different skills. Afterwards solutions are built based on experimentation. A very important aspect is also being able to transport solutions from other industries. Making associations between unrelated topics and understanding how things are done in a completely different environment can bring new inside…

It is very important that different departments (business, marketing, operations, maintenance, etc.) all work towards launching new innovations and removing obstacles because killing innovation is very easy, making it succeed is not.

Some good books on innovation are: The Innovator’s Dilemma, Nail It then Scale It and  The Innovator’s DNA…

Free, the disruptive French telecom operator and ISV, is changing the rules. Via Femtocell and via controlling the WiFi access points of its customers, Free is planning to offload a lot of mobile traffic via its fiber network. This is translated into very sharply priced mobile calling and data plans. Free’s Founder is telling the telecom industry they should no longer try to make money with communication but focus on identity and payment services.

Free is right to change the rules of the game instead of waiting for non-telecom disruptive players to do so. However what else could Free do to generate extra revenues?

Social Mobile Graph

Facebook is talking about social commerce in which friends, family and colleagues are taking an active role in your buying behaviour. At the moment social networks are either for business reasons, e.g. LinkedIn, or for pleasure, e.g. Facebook. However both need a lot of maintenance effort in which you need to send or accept invites from people who you might have known 20 years ago.

What if your calling and messaging behaviour could take away a lot of this burden? If you call somebody mostly during business hours then this person is likely to be a business contact, especially if other business contacts of yours have the same behaviour. Your addressbook and linkedin could be automatically updated. However you could go a lot further and see which restaurants your direct business contacts call more often. Anonymizing this information and creating public APIs and a marketplace for app developers could lead to a lot of innovative services that can be monetized.

Numbering Plan Apps

The numbering plan is probably one of the most under-used operator assets. However everybody knows how to dial a number. Why not let other people make new numbers, e.g. based on non-existing country codes or using the # or * combinations? People would be able to make premium services for everything from voting, surveys, competitions, money transfers, etc. Putting *120* in front of your number could mean that the caller is paying you 1,20 euros per minute to call you. It is up to you to redirect your number to an application that makes people want to call you. You might have a large numbering app market to choose from. Add a # and a number at the end and you could have thousands of applications behind one number. The operator would get a revenue share.

Call Center as a Service

Call centers are mainly used by large corporations. However small groups of ad-hoc people could benefit from them as well. Ad-hoc software support hot lines in which experts can be freelancers could be of interest to some. But it could even be as simple as housewives that can help you with recipes. As long as rating the participant’s value, dynamic joining and leaving of participants, paying participants a revenue share, configurable participant selection rules, etc. are provided, the applications are limitless.

A lot more

These are just ideas but there are a lot more possibilities that you can implemented. Especially if you can control both the mobile device as well as people’s access point. However the past has shown that trying to get a few people pay a lot of money for a service and operator’s trying to do it all by themselves, have not been successful. Innovation is not only needed in the product domain but also in the business domain. Models that should be explored are:

• Freemium, whereby most do not pay but get the traffic to your service and only a minority pay for advanced usage. Many examples in the web 2.0, e.g. LinkedIn, Zynga, etc.
• Long Tail, whereby not only a couple of high paying  groups are targeted but instead thousands of niches are targeted via the use of a general platform or third-party eco-system, e.g. Google Adwords, Facebook Apps, etc.
• Revenue Share, whereby others get the bulk of the revenue because they take the risk and the operator gets a small share but gets it from a large group of revenue sharers, e.g. Apple’s App Store

Telecom operators pay millions in software licenses each year. By doing so they are sustaining an industry of “feature loading”. “Feature loading” refers to complex software solutions that in order to win RFPs add more and more features. Most telecom operators are using RFPs to compare different software solutions. Whoever has more features for the lowest price wins the deal. The end result is that telecom software is unnecessary complex and expensive. Software providers do not want to respond with “not compliant” and prefer to add some extra feature even if the one who wrote the RFP will never ever use them.

The likes of Apple have shown us that software is most beautiful when it does very few things very well. The era of mini applications allows users to use special purpose “apps” for each activity. No training required. No heavy investment. No heavy integrations.

Telecom operators should move away from the long RFPs with hundreds of features being compared. Instead they should try to simplify. Why pay millions for a complex system that does too many things too complex? Many large dotcoms have moved away from this type of solutions and have used Open Source, have built single purpose systems/services or generic platforms with plugins to reduce complexity.

Examples:

• Amazon has pioneered Cloud Computing and has created individual single purpose systems or services that are easily accessible via REST or Web Interfaces. Different individual services (e.g. product recommendation, virtual server, virtual storage) get aggregated into complex solutions at the last moment.
• Google built its Google File System, BigTable, etc. as generic platforms on which hundreds of other services could be easily added.
• Thousands of dotcoms are using Hadoop, Cassandra, etc. to store data.

Each telecom service needs to be provisioned, rated, charged, billed, monitored, operated, supported, migrated, etc. By building solutions in which network, IT, communication and services are mixed into mega-complex architectures it has become impossible to launch new services in less than 12 months.

Building a Free Telco PaaS

How to do it differently? Is it possible to build a zero-license Telco PaaS that acts like a giant service delivery platform in the Cloud? YES

Operators will need to use Open Source, IaaS and SaaS solutions. IaaS can be delivered cheaply by using Open Source components: KVM for virtualization, Open Nebula for virtual machine and storage management, Hadoop/Cassandra for storage, Open vSwitch for network virtualization, etc. On top PaaS platforms can be built with solutions like WSO2 Stratos. Telecom services like Twilio‘s or the private cloud version, RestComm, can be used to allow developers to quickly create VAS. Open Source billing systems have been announced, like Meveo. Online shops can be build with Opencart. Datawarehousing and data analytics with Pentaho or Jasper Reports. There are hundreds of open source monitoring solutions: Icinga, Nagios, Zenoss, etc. Helpdesk can either be SaaS like Zendesk, or Open Source like Request Tracker. CRM like SugarCRM. SIP backoffice systems like FreeSwitch.

Operators should start thinking about the Cloud as a way to simplify internal integrations. All back-office systems should be shielded from the outside via easy to use REST, Thrift/Protocol Buffers, etc. interfaces. Service-based loadbalancing should allow service upgrades and rolling migrations without outages. The architecture should be built with Salesforce.com in mind. Non-programmers, and even better end-users, can build their own VAS by using drag-and-drop interfaces and combining different service blocks together into custom solutions. Plug-ins allow for custom behaviour without cluttering a solution for the rest of the users.

Operators should embrace new disruptive technologies to simplify their business, lower their cost structures and be able to launch new services every hour of the day. Large dotcoms are launching new features every day and use A/B testing to validate if users like them and they add to the bottom line. Marketing and product management get a totally different dimension…

2012 will be the year in which Apple’s mobile app revolution will be translated to every device (PCs, tablets, mobiles, signage, m2m, cars, TVs, etc.) and to the enterprise. Instead of a static company portal and an IT-driven software selection, 2012 will bring apps to the enterprise. Workers will be able to use their PC’s browser as well as a BYOD (bring your own device) to select apps from a company-wide or global enterprise app store. No longer will you have to pay for an annual license to edit a video, image or CAD drawing if you only use it twice a year. Software will be a lot more social. Not only IT will loose power, also marketing and upper management. Crowd-sourcing can allow employees to vote and rate and as such let content and opinions bubble up that might not always fit upper management’s strategy. However when used correctly the opinions are likely to beat any internal reporting system or dashboard in accuracy.

What is still pending?
Except for easy-to-use apps, inter-app and Backoffice integration is very important. Expect new “standards” based on innovative dotcom solutions in this area. Enterprise PaaS, a là Salesforce.com but often in private cloud, will move a lot of Excel and Access apps into SaaS apps. Employees will be the major enterprise app creators and no longer programmers.

Easy over training
This app revolution will focus on mini apps with basic functionality and no longer full enterprise solutions that do everything but in a too complex way.

Telecom’ s involvement?
What I described so far sounds like an IT platform and solution however it will span communication services as well. The link between IT and telecom will become very blurry. For this reason it is important for operators to be active in this market.