The Swisscom Voice Assistant
Riccardo Lopetrone • June 12, 2020
Switzerland's first Voice Assistant
The Swisscom Voice Assistant is unique as it can speak up to five languages, especially Swiss German. It has been developed by the Swiss telecommunication company Swisscom.
Voicetechhub speaks to Riccardo Lopetrone, Senior Product Manager Voice @Swisscom. We want to share these great insights with you.
Tell me more about your background, experience and how you got in touch with Voice?
I have been working as a TV Product Manager for different telecommunications companies for the last 15 years. Since 5 years I'm responsible for the customer experience around voice controlled devices @Swisscom.
My first contact with voice was in 2011, when Apple presented its first iPhone with Siri. I bought the new 4S model straightaway and made my first experiences with voice assistants. I found the feature very fascinating, even if the Siri capabilities were very limited back then. I then recognized that I wanted to deep-dive into the topic.
In 2016 at Swisscom we developed the first TV box with integrated voice control in Switzerland, a push-to-talk solution on the TV remote control which supported several voice commands on Swisscom TV. When Alexa was then launched in 2018 I started my first experiences with smart speakers. Today I have over a dozen devices at home (Amazon Echo, Google Nest, Homepod and of course the Swisscom Box), which I regularly use to e.g. play music, create shopping lists, control my smarthome devices or check the weather forecast.
Last year Swisscom decided to take the next innovative step so we integrated our own smart speaker into the new Swisscom Box.
How would you describe the Swisscom Box compared to other smart voice-enhanced devices?
All new Swisscom customers with a Swisscom TV contract get the new Swisscom Box with the integrated Voice Assistant. Existing customers can also upgrade to the new box. Beneath great entertainment features and a wide choice of content from the most popular providers (e.g. Teleclub, Netflix, Sky, etc.) the Swisscom Box comes along with two microphones and a speaker to support the new Voice Assistant. The Box is delivered with deactivated microphones to cover data privacy concerns. The customers need to actively switch on the microphones before they can use the voice assistant. Only then the Swisscom Box will react to our wake word "Hey Swisscom"
A few of the available skills (e.g. Spotify or the WiFi skill) first need to be activated on the separate companion app (Swisscom Home App) to allow account linking and for additional information around data privacy (e.g. when showing your WiFi password on the TV screen. All other skills are activated by default.
What are the main features?
Our focus was on the voicification of Swisscom products and services.
So far, the following skills are available:
- TV (e.g.channel change, title & actor search, trick modes, TV recommendations, etc.
- Smarthome (e.g. switch on and off your smarthome devices, execute scenes, etc.)
- MyCloud (voice control of your pictures and videos from the Swisscom cloud on the big TV screen)
- Weather (ask for the weather forecast in Switzerland, in collaboration with our companies local.ch & search.ch)
- News (ask for the latest news, in collaboration with our company bluewin.ch)
- Router & WiFi control (switch on and off your Internet & Wifi, present the WiFi password on the TV screen)
What is unique about the Voice Assistant of the Swisscom Box?
- Strong data privacy policies
- Speech recognition in 5 languages (DE/FR/IT/EN including a Swiss German model for the main dialects in Switzerland
- A deep integration into our services and products which would only have been partially possible on a 3rd party voice platform
What were the main challenges when developing the box?
- First of all we had a very challenging launch target. We developed all the components including the hardware in less than 12 months. Since many teams were involved (more than a dozen) the coordination was very complex.
- A further big challenge was the support of different language models and on top the huge amount of entities (terms) which come along with some of our skills. For the TV search for example we have more than 30'000 programme and movie titles which are available for our customers every week. And sometimes we need to support also mixed languages within the same language model. E.g. if you have chosen the English speech recognition but you are looking for movies with a French actor ("show me movies with Gérard Depardieu").
- Privacy and transparency around data storage was also a very important topic for us, so it took many discussions with our legal and data governance departments. As an example we decided to deliver the Swisscom Box with deactivated microphones so the customer first needs to activate them if he wants to use the voice assistant. Furthermore the customer can allow or decline that humans will listen to a small part of his generated audio files, which would allow us to improve the system.
What can we expect in the future?
In a first step we are optimizing the basic functionalities (e.g. the wake-word sensibility, the speech recognition for all the supported languages, the intent recognition, etc.). In parallel, we want to improve the existing skills and include more context to enhance the customer experience. And of course we want to expand the number of supported skills with further Swisscom services and products.
Will it be possible to develop a third-party application?
For the time being we are focussing on Swisscom Use Cases. We will evaluate in a later stage if it makes sense to open up the platform for other Swiss companies.
Riccardo Lopetrone
Senior Product Manager Voice
Riccardo Lopetrone is Senior Product Manager Voice at Swisscom, the leading telecommunications company in Switzerland. For more than 5 years he has been committed to creating great user experiences for voice controlled services.
Need support with your Generative AI Strategy and Implementation?
🚀 AI Strategy, business and tech support
🚀 ChatGPT, Generative AI & Conversational AI (Chatbot)
🚀 Support with AI product development
🚀 AI Tools and Automation
AI-powered chatbots, whether developed in-house or deployed through trusted platforms, are revolutionizing customer service, knowledge access, and internal communication. However, alongside these opportunities come new legal obligations: data protection , transparency , and EU AI Act compliance must be addressed carefully. This article covers: Where AI chatbots bring business value What compliance risks you must manage How to implement AI chatbots successfully and securely
What's RAG? The goal is for the language model is not to draw on its own knowledge (from the model), but for information to be enriched in the prompt. This is usually your own data you provide to the model (PDFs, systems).
Video Creation: The Ultimate Guide to Runway, Luma AI, Haiper.ai, and Hailuo AI
What are the best AI powerpoint tools. Discover tools that create presenations with AI.
Ultimative review of all relevant image creation tools
Optimizing your Website for AI: How to get found by ChatGPT. This article provides concrete Large Language Model Optimization strategies for SMEs and companies.
Since August 2024, users have been able to use the web version of the image creation tool Midjourney. This simplifies usage by providing a user-friendly interface to experiment with one of the top Generative AI image creation tools available. We tested it for you and are sharing helpful tips and tricks. How to prompt images with Midjourney? If you use Midjourney on discord, there is a clear prompt structure and prompt parameters to adhere to. Usually, it makes sense to stick to it: 1) To prompt use "/Imagine" 2) Then enter your subject (description and details) you want to see on the image and it's environment (see yellow highlighted below in the prompt example) 3) Then enter composition, lightning, colours (see green highlighted below in the prompt example) 4) Finally add technical parameters to adjust and finalize your image. Please find a useful parameter library here.
This article explains agentic AI and why it is so important when building generative AI and chatbot applications. Overview about Agentic AI vs. Gen. AI vs. RPA and all you need to know about these terms.
LLM Benchmarks: Finding the right LLM for your Needs
RAG vs. Finetuning. Open Source vs. Proprietary Models. We explain what makes sense when.
