“Alexa, activate the lights”; Lockheed Martin, Amazon, Cisco check digital assistant, video conferencing in house – NASASpaceFlight.com

With the Orion spacecraft hovering via its mission with flying colours to this point, the capsule will quickly enter its distant retrograde lunar orbit with important testing of assorted capsule and repair module methods progressing.

Inside Orion, along with the myriad experiments and expertise demonstrations, groups from Lockheed Martin, Amazon, and Cisco have been busy working via the Callisto demonstration. Callisto – a mixture of Alexa and WebEx video conferencing software program – goals to allow the presence of digital assistants and dependable video conferencing on future human spaceflight.

In an interview with NASASpaceflight, Rob Chambers – Director of Technique and Enterprise Improvement for Business and Civil Area, Lockheed Martin – mentioned: “From a human situation perspective, I am on the opposite facet of the moon, I am bodily distant, I am psychologically distant. My bandwidth is restricted. I’ve delays that may solely get longer as I transfer on to Mars.”

“How can we enhance consciousness and interactivity, make folks extra environment friendly, cease losing mind cells on the easy issues that computer systems can do?”

On Earth, one reply to those questions is the digital assistants constructed into our sensible expertise. So how about integrating it with Orion and future deep house exploration missions?

That is what Callisto is designed to check… and way more.

Alexa collaborating within the Callisto demonstration can not work together with any essential spacecraft methods akin to life help or propulsion. “Alexa can not cancel the mission or hearth an engine,” Chambers famous. “And rightly so.”

What Alexa might be examined for on Artemis I falls into the classes of controlling lights contained in the Orion capsule (which responds to voice instructions), correctly accessing spacecraft particulars to reply with appropriate data when requested, and appearing as a part of a digital presence gadget for video convention/whiteboard a part of the demonstration.

In whole, there are greater than a thousand created utterances that the workforce constructed to coach Alexa to grasp the place to search for data inside Orion for native data requests.

“For those who’re on the spacecraft, Alexa must know to go to the Orion pace and telemetry stream,” Chambers famous.

An enormous a part of that is that Alexa wants to have the ability to perceive that completely different phrases can be utilized to ask for a similar data.

For instance, “What’s my pace?” and “How briskly am I going?” are two alternative ways of asking for a similar knowledge. However when an astronaut on Orion asks any of those questions, they’re truly asking “How briskly is Orion going?” Alexa wants to have the ability to acknowledge the that means behind the question after which know which of Orion’s telemetry streams to entry to seek out the specified content material.

Oh, hi there Commander Campos! 👋

We took this image through the flight @NASA_Orions inside utilizing the entrance digital camera on @Apple iPad which is built-in in our #Callisto payload—taken about 50,000 miles from the moon!

Learn extra about Callisto: https://t.co/rTW8acim6R pic.twitter.com/YAcgWK3I4U

— Lockheed Martin Area (@LMSpace) November 20, 2022

All of that is simpler mentioned than performed, particularly in a spacecraft — which is not the most acoustically pleasant surroundings for a digital assistant like Alexa.

That aspect is examined day by day via using a number of completely different folks verbally speaking with Alexa in Orion from Mission Management during times of Deep Area Community (DSN) connectivity and when mission bandwidth permits.

A part of this can even contain ensuring Alexa can hear correctly within the not-acoustically excellent Orion capsule and may reply accurately to completely different voice patterns and accents.

One other facet of consideration is what occurs if the requested data can solely be collected by connecting again to the Web by way of the DSN. And what if Orion is out of sight with DSN at the moment?

One other space of ​​Callisto testing consists of what Alexa will say during times of Orion-to-Earth communication outages, together with indicating the time when the blackout will finish and the query may be answered.

Alexa’s means to work together with the web from lunar orbit can be being examined, as is its latency given the few seconds it takes alerts to journey the space between Earth and Orion.

“If we have been on the opposite facet of the moon, it is a few seconds spherical journey. It is not an enormous deal. However then you may have the switching, you may have all of the interfaces and handoffs within the methods,” famous Chambers.

“So if I requested a query, it then has to come back again right down to Earth to course of, pull the knowledge from the cloud, ship it again to Alexa, after which let Alexa formulate the outcomes. It may very well be like a 10-second spherical journey.”

This delay is ruled by the legal guidelines of physics, however understanding the response of human customers to it now may also help practice astronauts for future missions.

The human behavioral response to that delay additionally arms person interface designers with priceless details about what further behaviors the digital assistant can carry out, akin to enjoying music or beeping whereas it processes data through the delay, to permit the human person to grasp what it’s doing.

However one other vital a part of the Callisto demo with Alexa is video conferencing. And to do this with the obtainable bandwidths required “cutting-edge picture compression” on Cisco’s half for the WebEx system.

“We’re making an attempt to do, in some methods, trendy video conferencing over dial-up speeds. It is not that sluggish, however [we’re] speaking tens of kilobytes or a whole bunch of kilobytes,” Chambers mentioned.

For this check, Alexa herself should act as a digital crew member by way of inputs from Mission Management which are separate from the comparatively easy “speak to Alexa” checks.

That is the place the usual iPad commonplace comes into play, with Mission Management in a position to check video feeds, picture compression, choppiness, time lag and the accuracy of video-over-voice precedence by simulating a digital crew member via Alexa and the iPad.

The video convention checks are scheduled for 2 hours on days when the workforce has entry to the 70-meter dishes within the Deep Area Community. These dishes present the best potential bandwidth for the checks as Orion cruises the furthest a human-capable spacecraft has ever traveled from Earth.

“So the way in which it really works is in Mission Management the place we have now the Operations Management Heart, you will be sitting in entrance of what we name a desktop skilled. It is the Cisco setup. And it has a video digital camera,” famous Chambers.

“After which on board we have now the speaker and Alexa. And beneath that’s the iPad mounted. So whenever you speak in mission management, what you see within the spacecraft is Alexa and the iPad, and whenever you speak to Alexa, or she talks again, the blue ring will shine and your face, after all, might be on the iPad.”

The fourth main check for Callisto can be linked to video conferencing and bit fee compression within the type of interactive whiteboards between the Orion crew and controllers in Houston.

Make room for everybody!

By @NASAArtemis I mission, we invite visitors to the Mission Management Heart to check the applied sciences built-in into the #Callisto payload flying on board @NASA_Orion.

Keep tuned…👀https://t.co/pomlCY9sCd pic.twitter.com/tcjTZBgBTR

— Lockheed Martin Area (@LMSpace) November 19, 2022

“Allow us to say [you’re in Orion and] you add an image of the moon for the touchdown website,” Chambers associated. “You pull up the image. With [the crew’s] mild pen, [the crew] circles the place they’ll land. Right here on Earth you delete it after which spotlight and zoom in.”

“So it is an interactive whiteboard. Now it has that point lag; we will not repair the physics. However on the subject of interacting and collaborating and speaking about ‘following this specific path,’ an image is value a thousand phrases.”

Chambers continued, “So we are able to check it and make sure that there are not any errors, that it is clean. It is examined all through the digital community. The algorithms and the protocols are the trick.”

Total, the Callisto demonstration goals to show and acquire knowledge on how widespread types of communication may be superior in house exploration, not solely to permit the identical comforts and familiarities right here on Earth to be prolonged to house, but in addition for important missions. essential communication.

Moreover, the video conferencing aspect of the demonstration will also be utilized on Earth, the place authorities, media and protection operations can all profit from compressed bit charges whereas sustaining picture high quality.

(Lead picture: The Callisto demonstration in entrance of the ‘Commander’ model contained in the Orion spacecraft throughout Artemis I. Credit score: NASA)