Introduction to Voice Assistant

It has been a few years since Amazon unveiled the Alexa-powered Echo, but since then, smart speakers have become a major consumer-electronics category. Key to its success is the notion of the always-on virtual assistant, which other companies like Apple and Google have adopted as well. In fact, not only has Google made Assistant the driving force behind its Android smartphones, it has launched its own line of Echo rivals.

What is Mycroft AI

Mycroft is the world’s first open source voice assistant. It can run anywhere – on a desktop computer, inside an automobile, it even runs on a Raspberry Pi. It is open so it can be remixed, extended, improved. It can be used in anything from a science project to an enterprise software application.

You start using a Mycroft to set reminders, play the news, or turn the lights on, it’s hard to go back.  you get all the convenience of voice control without the data-collecting tech giant behind the scenes, and you can run it right on a Raspberry Pi.

Mycroft has been around for quite a few years, but it’s recently gained a bit more notoriety thanks to privacy concerns surrounding data collection at Amazon and Google. Unlike those assistants, Mycroft only collects data if you opt in during setup. And for the users who do opt in, Mycroft promises “never to sell your data to advertisers or third parties“—instead, it only uses it to help developers improve the product. Mycroft even uses the privacy-focused DuckDuckGo as its search engine instead of Google when you ask for information.

History of Mycroft

Inspiration for Mycroft came when Ryan Sipes and Joshua Montgomery were visiting the Kansas City makerspace, where they came across a simple and basic intelligent virtual assistant project. They were interested in the technology, but did not like its inflexibility.Montgomery believes that the burgeoning industry of intelligent personal assistance poses privacy concerns for users and has promised that Mycroft will protect privacy through its open source machine learning platform.

Mycroft has won several awards including the prestigious Techweek’s KC Launch competition in 2016. Mycroft was part of the Sprint Accelerator 2016 class in Kansas City and joined 500 Startups Batch 20 in February 2017.The company accepted a strategic investment from Jaguar Land Rover during this same time period.To date, the company has raised more than $2.5 million from institutional investors and has opted to offer shares of the company to the public through Start-engine, an equity crowdfunding platform.

How Mycroft AI works?

Behind any Voice Assistant lies the power of Speech processing. Natural Language Processing (NLP)

Voice Assistant Work-flow

Any Voice Assistant has to do 3 basic steps

  1. Listen to the wake word. Once a wake word is identified start the next step other wise go back to the loop to identify wake word.  
  2. Once the wake word identification is successful, Voice to text Processing, Full NLP Processing and query/Command Execution happens. There are 2 outcome of this step
    1. Command to Mycroft.(start timer, switch on lights etc)
    2. Query formation which requires internet access(Play music on spotify)
  3. Once the command or query execution is is done , then send the response(text).
  4. Process the text response via text to speech.

There is complete Block of Hidden Markov Model (HMM) and Speech processing engine to do in step 2.

Mycroft Workflow

Mycroft how it works

Just like any other Voice assistant Mycroft uses voice recognition to process the voice commands.
Speech recognition involves the complex NLP involved in processing.

1st step is to identify the wake word.

Wake word processing is done by HotWordEngine.
Like its name suggests, a Wake Word Listener’s job is to continually listen to sounds and speech around the Device, and activate when the sounds or speech match a Wake Word. Unlike other hotword/Wakeword detection products, Mycroft Precise is fully open source.

PS : Hotword and wakeword are the same, mycroft-source code refers it as hotword.

Mycroft Supported HotwordEngine

Mycroft has 4 supported hotwordEngine

  1. Precise : Opensource developed by Mycroft
  2. PocketSphinx: Designed by CMU research State of art speech recognition algorithms for efficient speech recognition.
  3. SnowboyHotWord: Snowboy is a 3rd-party hotword engine providing an easy training and testing interface.
  4. PorcupineHotWord: Another 3rd-party lightweight wake word engine. It enables building always-listening voice-enabled  applications. The application is developed by Picovoice. This wake-word engine is opensource

After Wake word mycroft starts sampling the audio.
Post sampling Speech Recognition is used to process the commands. For Speech processing commonly referred as NLP
SpeechRecognition python package is used.

  1. Sampling of audio.
  2. Conversion of Audio to text(speech to text) here Heuristic Markov model is used…
  3. Intent processing.: Intent Processing leads to query
    Query is further categorized into 3 types

Does Mycroft Needs internet.

NO,Mycroft can operate without internet,. But it will limit its functionality.

So, without internet You will not be able to

  1. Play songs from Spotify
  2. Ask: What is the height of Everest or Eiffel tower.
  3. Check your Calendar.
  4. Call a uber

What you can do without internet:

  1. Set a timer
  2. Control your smart home sensors
  3. Access your Smart-Security Systems
  4. Give commands to control lighting, may be switch on/off a smart plug.
  5. Switch on the tv (But you may not ask to play Netflix)

If you are developer you can access the Code here. Best part is it is developed in Python!!!

If you are interested in building a Mycroft Voice Assistant, follow the link below.

DIY Mycroft on Raspberry Pi 4

if you liked the article, or want to provide any suggestions , please write below in comments.

3.3 3 votes
Article Rating
Previous articleSmartNICs…. Reducing the Networking workloads
Next articleUnderstanding Mesh wifi
Notify of
1 Comment
Newest Most Voted
Inline Feedbacks
View all comments
Mona lisha
Mona lisha(@monalisa)
1 year ago

Good in depth article