Robot plays „Rock, Paper, Scissors“ – Part 1/3

Gesture recognition with intelligent camera

I am passionate about technology and robotics. Here in my own blog, I am always taking on new tasks. But I have hardly ever worked with image processing. However, a colleague’s LEGO® MINDSTORMS® robot, which can recognize the rock, paper or scissors gestures of a hand with several different sensors, gave me an idea: „The robot should be able to ’see‘.“ Until now, the respective gesture had to be made at a very specific point in front of the robot in order to be reliably recognized. Several sensors were needed for this, which made the system inflexible and dampened the joy of playing. Can image processing solve this task more „elegantly“?

Rock-Paper-Scissors with Robot Inventor by Seshan Brothers. The robot which inspired me for this project

From the idea to implementation

In my search for a suitable camera, I came across IDS NXT – a complete system for the use of intelligent image processing. It fulfilled all my requirements and, thanks to artificial intelligence, much more besides pure gesture recognition. My interest was woken. Especially because the evaluation of the images and the communication of the results took place directly on or through the camera – without an additional PC! In addition, the IDS NXT Experience Kit came with all the components needed to start using the application immediately – without any prior knowledge of AI.

I took the idea further and began to develop a robot that would play the game „Rock, Paper, Scissors“ in the future – with a process similar to that in the classical sense: The (human) player is asked to perform one of the familiar gestures (scissors, stone, paper) in front of the camera. The virtual opponent has already randomly determined his gesture at this point. The move is evaluated in real time and the winner is displayed.

The first step: Gesture recognition by means of image processing

But until then, some intermediate steps were necessary. I began by implementing gesture recognition using image processing – new territory for me as a robotics fan. However, with the help of IDS lighthouse – a cloud-based AI vision studio – this was easier to realize than expected. Here, ideas evolve into complete applications. For this purpose, neural networks are trained by application images with the necessary product knowledge – such as in this case the individual gestures from different perspectives – and packaged into a suitable application workflow.

The training process was super easy, and I just used IDS Lighthouse’s step-by-step wizard after taking several hundred pictures of my hands using rock, scissor, or paper gestures from different angles against different backgrounds. The first trained AI was able to reliably recognize the gestures directly. This works for both left- and right-handers with a recognition rate of approx. 95%. Probabilities are returned for the labels „Rock“, „Paper“, „Scissor“, or „Nothing“. A satisfactory result. But what happens now with the data obtained?

Further processing

The further processing of the recognized gestures could be done by means of a specially created vision app. For this, the captured image of the respective gesture – after evaluation by the AI – must be passed on to the app. The latter „knows“ the rules of the game and can thus decide which gesture beats another. It then determines the winner. In the first stage of development, the app will also simulate the opponent. All this is currently in the making and will be implemented in the next step to become a „Rock, Paper, Scissors“-playing robot.

From play to everyday use

At first, the project is more of a gimmick. But what could come out of it? A gambling machine? Or maybe even an AI-based sign language translator?

To be continued…

modl.ai closes €8.5m Funding for AI Engine to Unleash Bots and Transform Game Development

COPENHAGEN, Denmark—November 2022—modl.ai, which seeks to remove the shackles of game development with a transformative AI engine, announced today it has closed Series A funding of €8.5m with lead investor Griffin Gaming Partners and M12 – Microsoft’s Venture Fund. The company received additional participation from Rendered.vc, PreSeed Ventures, and Transistormedia.

The modl.ai AI engine is redefining game development by equipping developers with an essentially unlimited army of bots that can adapt to various playing styles, employ dynamic moves and tactics. This gives developers power to enhance automation and remove the redundancies and manual efforts that often stymie and delay the release of new games and updates by months or even years.

Pierre-Edouard Planche, Partner at Griffin Gaming Partners, said: „Gaming is one of the trickiest user experiences for which to automate testing, and it therefore requires an utmost degree of technical talent and research combined with direct experience working in games. This is why we are very excited to be backing Christoffer, Lars, Benedikte, and the modl.ai team to help streamline a growing pain and multi-billion-dollar industry expense that is top of mind for many game developers.“

Carli Stein of M12 added: “At M12, we’re excited about the impact modl.ai will have on the gaming industry, allowing game developers to focus on the parts of development they love, while automating the costly, time-consuming, and manual processes they don’t,” said Carli Stein, an investor at M12. “When we met Christoffer and the modl.ai team, we knew they had the right combination of highly technical domain expertise, coupled with a product suite that has the potential to revolutionize game development as we know it.”

modl.ai was founded by a talented team of game developers, engineers and AI experts led by CEO Christoffer Holmgård, who saw the opportunity to use AI to automate and improve multiple aspects of the game development process. Between them, the company founders have developed and launched more than 30 games for PC, Mac, Playstation, Xbox, Switch, Apple Arcade and further platforms.

“We started modl.ai to build the tools and technology we always wished we had as game developers,” said Christoffer Holmgård, CEO and Co-Founder. “In 2018, we decided that the timing was right, both for technology and the market. The founders had worked together in games and AI for more than a decade, always discussing how we could bring advanced AI to game developers broadly, offering something completely new to the industry that we love. All games are unique, creative works, but they share production needs and technological characteristics. That’s why the games industry is becoming increasingly modular and standardized. This is a good thing, and we want to be the de facto AI Engine for as many games as possible.”

With the new funding, the company will expand its reach to make the AI Engine available to developers worldwide. With simple setup, developers will get access to AI players – to not just test and balance their games but also have access to customizable bots who will play their game with or against their player base. modl.ai’s AI Engine works with the major publicly available game engines such as Unity and Unreal Engine and can be extended to support any modern game engine.

modl.ai’s AI Engine drives bots for games, enabling AI players that test, evaluate, and even play games in the place of human players. The market for testing and quality assurance in games alone is several billion dollars, and the need for bot players has become crucial in a multiplayer focused $200Bn+ industry.

The games industry is on the cutting edge of technology and innovation; however much of the game development process is highly manual. This means significant creative potential with highly skilled developers, as well as precious speed to market, is lost to rote work and monotonous daily routine. modl.ai is on a mission to change this, building an AI Engine that frees developers to focus on creating, by automating the parts of the game development process that are repetitive and time consuming through AI players.

„Game testing is an endemic, universal, yet overlooked pain point in the industry that has material impact on studio operations, profitability, and title performance. Automation solutions have not emerged due to the complex nature of games and unique design. We strongly believe that modl.ai is the breakthrough team to tackle this moonshot opportunity and, in doing so, is positioned to build a powerful AI platform developed for 3D environments that can grow beyond games,“ stated Jiten Dajee, Rendered VC.

“At PreSeed Ventures, we know it’s all about a few individuals with a rare set of traits and the right mentality to move the needle and modl.ai’s CEO, Christoffer Holmgård is one of those people,” said Mads Klarskov Petersen, COO of PreSeed Ventures. “modl.ai has not only attracted top talent within the space, 12 PhDs from applied AI in games, but has proven the growing importance that AI plays across different aspects of modern game development. Moreover, Christoffer is one of our favorite examples of how a Danish startup can attract international talent and investors, by building a unique technology and sticking to its purpose: helping developers to enhance and increase player engagement no matter the market direction.”

ABOUT modl.ai

Headquartered in Denmark, modl.ai is a four-year-old company that is transforming the game development market with its state-of-the-art AI Engine and machine learning models. Staffed by many of the industry’s best and brightest, modl.ai currently ranks second in the world among private companies for the number of technical game publications its team has collectively authored. With modl.ai’s engine and game-playing bots, testing, evaluation, overall game development and speed to market will never be the same. „If anyone can bring games AI research to the industry and succeed, this is the team“ – Ken Stanley, OpenAI alum.“ For more please visit modl.ai.

igus igumania game

Igus igumania game. Build your own Mars Rover assembling automation factory and improve it with @igusgmbh products. Enjoy this new game soon in your Webbrowser and on other plattforms. Get to know Rusty the robot and Dave the igus employee while learning about igus smart plastics products and low cost Automation. I really enjoyed playing this game, mainly for one reason: robots 😉

Mit der Programmier-App »NEPO Missions« auf Mars-Expedition gehen

Nach neuem Nutzer*innen-Rekord auf der Open-Source-Programmierplattform »Open Roberta« startet Fraunhofer IAIS jetzt eine neue Lernspiel-App, basierend auf einer Mars-Mission der ESA.

Programmieren wie die Profis, forschen wie die Weltraumfahrer*innen – die Spiele-App »NEPO Missions« ermöglicht beides. Das Fraunhofer IAIS veröffentlicht ein Lernspiel, in dem Kinder und Jugendliche spielerisch einen virtuellen Mars-Rover programmieren können, der auf »Rosalind Franklin« basiert: Den realen Rover will die Europäische Weltraumorganisation ESA auf die Forschungsmission »ExoMars 2022« schicken. Die Idee zu der App hatte sich 2018 bei der »Google.org Impact Challenge« durchgesetzt und wurde als Leuchtturmprojekt gefördert. NEPO Missions basiert auf der Fraunhofer-Plattform »Open Roberta Lab«, mit der allein in 2020 rund drei Millionen Menschen weltweit ins Programmieren eingestiegen sind.

Digitale Kompetenzen praxisnah stärken – das ist seit mehr als 18 Jahren Ziel der Initiative »Roberta® – Lernen mit Robotern« des Fraunhofer-Instituts für Intelligente Analyse- und Informationssysteme IAIS. Mit der zugehörigen Open-Source-Programmierplattform »Open Roberta® Lab« ermöglicht das Team aus Didaktik- und Robotik-Expert*innen Menschen weltweit, ihre Hardware kostenfrei zu programmieren. Mit rund drei Millionen Zugriffen auf lab.open-roberta.org allein im Jahr 2020 hat sich die Nutzer*innenzahl aus dem Vorjahr versechsfacht. Mit einer neuen Programmier-App für Kinder ab zehn Jahren setzt die Initiative den bisherigen Erfolgsweg von Open Roberta fort: »NEPO Missions« ist ab sofort kostenfrei im Google Play Store und im App Store verfügbar.

NEPO Missions setzt auf das Konzept des visuellen Programmierens, das im Open Roberta Lab seit 2014 Kindern ab dem lesefähigen Alter den spielerischen Einstieg ins Programmieren ermöglicht. Per »drag and drop« werden die Blöcke der grafischen Fraunhofer-Programmiersprache »NEPO« zusammengesteckt und schließlich von realen oder simulierten Robotern und Mikrocontrollern ausgeführt.

Reale ESA-Forschungsmission »ExoMars 2022« dient als Inspiration

Wie im Open Roberta Lab können Nutzer*innen auch mit der neuen Lernspiel-App NEPO Missions simulierte Roboter programmieren – jetzt allerdings in einem ganz besonderen Szenario: Die Mission bringt Spieler*innen auf den Mars. Sie erhalten den Forschungsauftrag, mithilfe eines ferngesteuerten Rovers die Oberfläche des Planeten zu erkunden. Schritt für Schritt lösen sie dafür immer schwierigere Programmieraufgaben: vom Fahren auf der Mars-Oberfläche über das Sammeln von Daten mittels Sensoren bis hin zu Bohrungen, um tiefere Bodenschichten zu untersuchen. Wie im Open Roberta Lab stellen die Kinder die Programme intuitiv mit den grafischen NEPO-Blöcken zusammen und erzielen im Handumdrehen erste Erfolgserlebnisse.

Die Mission ist an die reale Forschungsmission »ExoMars 2022« der Europäischen Weltraumorganisation ESA angelehnt. Bei dieser soll der Mars-Rover »Rosalind Franklin« den »roten Planeten« auf biologische Aktivitäten untersuchen und so bestenfalls Spuren von Leben finden. Im Spiel NEPO Missions programmiert der*die Nutzer*in als fiktives Mitglied der ESA eine virtuelle Version des Mars-Rovers. Insgesamt 17 Programmieraufgaben sowie Hinweise und Tipps erhalten die Spieler*innen von dem Roberta-Maskottchen sowie der fiktiven Teamleiterin Dalia, die gleichzeitig eine weibliche Vorbildfunktion einnimmt.

Start zum Internationalen UNESCO-Tag der Frauen und Mädchen in der Wissenschaft

Das Fraunhofer IAIS startet die neue Coding-App passend zum Internationalen Tag der Frauen und Mädchen in der Wissenschaft der UNESCO am 11. Februar 2021. Auch, wenn sich das Lernspiel an beide Geschlechter richtet, ist es seit jeher Ziel der Roberta-Initiative, insbesondere Mädchen für MINT zu begeistern. Dieses Ziel teilt Roberta sich mit Dr. Carmen Köhler von der Initiative »Erste deutsche Astronautin« gGmbH. Sie erforscht in mehrwöchigen internationalen Mars-Simulationen als »Analog-Astronautin« vom Österreichischem Weltraumforum marsähnliche Gebiete auf der Erde. In Zusammenarbeit mit Fraunhofer IAIS bietet sie auch »Code4Space«-Workshops für Kinder und Lehrkräfte an. »Als Analog-Astronautin weiß ich, wie wichtig es ist, in realistischer Umgebung zu trainieren. Man muss meist weit reisen, um marsähnliche Gebiete zu finden. Mit dem Spiel NEPO Missions bekommen Kinder schon im frühen Alter einen realistischen und spielerischen Zugang zum Programmieren und können sich auf ihre Zukunft als Weltraumforscher*innen vorbereiten, egal ob im Wohn- oder Klassenzimmer«, sagt Carmen Köhler.

Die enge Verknüpfung des Lernspiels mit einer realen Mission vermittelt Kindern digitale Kompetenzen sowie einen wirklichkeitsnahen Einblick in die Arbeit von Wissenschaftler*innen. »Als Serious Game dient das Spiel nicht nur der Unterhaltung«, sagt Thorsten Leimbach, Geschäftsfeldleiter Smart Coding and Learning, dessen Team am Fraunhofer IAIS die App in Zusammenarbeit mit dem Serious-Games-Hersteller »Gentle Troll« entwickelt hat. »NEPO Missions verbindet die Faszinationen des Weltraums und der Technik mit einem echten wissenschaftlichen Abenteuer. Mit dem Spiel möchten wir den Forschungsgeist in Kindern wecken und ihnen Mut machen, ihre digitale Welt selbst zu gestalten.«

Künstliche Intelligenz im Einsatz – mit Fraunhofer-»Text-to-Speech-Technologie«

Neben dem intuitiven Programmieransatz kommt in NEPO Missions auch eine Text-to-Speech-Technologie zum Einsatz, die den Spieler*innen Aufgabenstellungen und Informationstexte innerhalb des Spiels vorliest. Im Rahmen des vom Bundesministerium für Wirtschaft und Energie (BMWi) geförderten Projekts SPEAKER rund um eine Sprachassistenzplattform »Made in Germany« haben die Expert*innen des Fraunhofer-Instituts für Integrierte Schaltungen IIS diese Technologie in Kooperation mit dem Fraunhofer IAIS entwickelt. Beide Institute gehören zu den europaweit führenden Forschungseinrichtungen für die angewandte Forschung rund um Künstliche Intelligenz.

NEPO Missions richtet sich an Kinder ab 13 Jahren, unter Aufsicht von Erwachsenen ist auch eine Nutzung ab zehn Jahren möglich. Das Spiel wird ohne In-App-Käufe als Open-Source-Projekt kostenfrei bereitgestellt und ist ab sofort zum Download im Google Play Store und im App Store verfügbar. Die App ist kompatibel mit folgenden Betriebssystemen und neueren Versionen: iPhone iOS 14, iPad iOS 14 und Android 6.0. Die Finanzierung und Entwicklung des Lernspiels wurde mithilfe der »Impact Challenge 2018« von Google.org, dem philanthropischen Arm des Unternehmens, ermöglicht. Dabei wurde die Idee zu NEPO Missions als eines von zehn Leuchtturm-Projekten ausgezeichnet und mit 250 000 Euro gefördert.

Dass der praxisnahe Ansatz der Roberta-Initiative gut ankommt, belegen auch die stetig wachsenden Nutzungszahlen im Open Roberta Lab. »Von Brasilien über Rumänien bis Japan – aus mehr als 120 Ländern programmieren Kinder und Jugendliche aber auch Auszubildende und Manager*innen auf unserer Plattform ›made in Germany‹«, erklärt Thorsten Leimbach. Die Programmier-Plattform wird am Fraunhofer IAIS in Sankt Augustin unter höchsten Datenschutz-Maßnahmen entwickelt. Open Roberta ist als Open-Source-Projekt auf GitHub für jeden frei einsehbar. Seit 2020 werden zudem keine Cookies mehr auf der Plattform gesetzt.

Weitere Informationen:

Mixbots, A Robot Building Trading Card Party Game Now On Kickstarter

Funamite LLC launches Mixbots, a robot building trading card game on Kickstarter. Robots are built by applying stickers onto trading cards, and combat is played with the hand game, Rock Paper Scissors. Combat is fast and furious for one on one duels and team battles with 10 or more players. Mixbots is enjoyed by tweens, teens, new gamers, and groups of friends.


Building robots is half the fun and putting them in action against other robots completes the whole experience. After building their robots, players will have a chance to personalize them by writing a name on the back. In team battles, players can help their own teammates by using their robots’ special abilities. “Students were on the edge of their seats. They were very engaged and enjoying the game”, said Christine Ha, a middle school teacher.

Funamite decided on using stickers instead of transparent cards to build the robots because that allowed the player’s creations to remain intact when being carried about, and to easily be seen. When multiple transparent cards are stacked on top of one another, details become blurry, so stickers were the easy choice. Funamite decided on Rock Paper Scissors as the combat mechanism because it’s already a familiar system with many and therefore lowers the learning curve and barrier to entry.

Funamite LLC is game publishing company founded in 2016 and based in Elk Grove, California. Mixbots is the company’s first product.

The Mystery of Robot Planet – 8Bit Game on Indiegogo

I just received this message from Danni, creator of the 8Bit game „The Mystery of Robot Planet“:

The Mystery of Robot Planet is an 8bit inspired adventure-puzzle game for Windows, OS X and Linux scheduled for release in 2018. Visually inspired by early Pokemon games and mechanically based on other adventure-puzzle games such as Monkey Island and Beneath a Steel Sky, the game follows the player character, Ivan, on his quest to become a Marine and save the Princess!

 

With many vibrant locations, planets and separate endings depending on choices made throughout the game, The Mystery of Robot Planet aims to be a fun, unique and immersive experience for those who crave the puzzle-solving goodness of the early 90s point and click adventure games.

 

Download the Playable Teaser here – https://roadbloc.itch.io/the-mystery-of-robot-planet
Donate on IndieGoGo – https://www.indiegogo.com/projects/the-mystery-of-robot-planet-game-adventure/x/4781251#/

 

PBL ROBOTS dice & card game

William Hessian and John „Sighless“ Supinski have spent the last year creating a brand new strategy card and dice game called PBL ROBOTS! „The game is a interesting mix of Risk and Magic the Gathering, except this game has giant 100 foot Robots!“ William said. The card game challenges each player to collect cards in a draft format in order to build their deck. When the battle begins each player slide their pilot card into the robot card, courtesy of a die cut cockpit in the center of the robot. Then players lay armor cards on the robot to watch the robot grow stronger and boost up the attack and defense. When players start attacking they roll dice to determine if the robot will blow of an arm, a leg or a head. Each pilot has a powerful critical form when you are down to one remaining limb, which gives the losing player a chance to make a comeback, but when a robot has lost of four limbs the game is over.

William and John have spent their free time at their apartment on the East End of Portland, Maine developing this intense card game consisting of 113 unique cards and 3 different game play modes. William is a local visual artist and activist, working at Preble Street, teaching at Rivertree Art in Kennebunk and volunteering at the Meg Perry Center. William did all of the artwork and design for the game and originally invented the concept for the game over ten years ago in Minnesota, but did not start finishing it until meeting Sighless in 2012. Sighless is a local musician in the Bumbling Woohas and works at the Maine People Alliance, and was the host of Turnstyle Thursday open mic at CTN Channel 5 for over a year. Once Sighless began working on the game he created a business plan, invented characters and began writing back stories. A group of artists called the Hidden Ladder Collective, which William and John helped start, began play testing the game in-between hosting art shows, art making gatherings and busking out on the streets. Other members of Team Robots, Abbeth Russell and Kara Oster made important contribution to the game.

Thursday, October 23rd the collective has launched the Kickstarter hoping to raise the $22,000 they need to print the first edition versions of the game. As of November 1st they have raised $7,000 towards their goal. Over 100 letters have gone out to game stores across the country. William and Sighless and PBL ROBOTS fans have been playing PBL ROBOTS demo events weekly at R Choice 2 Gaming Thursday Nights which is a free event open to the public. The group has been travelling to Louisana, New Orleans and the Midwest to demo the game.

While the game itself is optimal for players 12 years and older, due to the high level of strategy, the game is also very appealing to a younger audience. William teaches art at Rivertree Arts in Kennebunk and his 4th to 6th grade students have loved the PBL ROBOTS cards. „The kids come back the next day demanding to have another glance at the game, and their parents follow right behind curious to see what the kids are so excited about,“ William remarked. Because of the kids persistence William and Sighless invented a new way to play the game that makes it a little easier for a younger audience, but retains the fun and excitement of the longer game. The popularity of the game is already growing with over 900 Twitter followers and over 200 Facebook likes.

To check out their Kickstarter visit their website at: https://www.kickstarter.com/projects/robotcardgame/pbl-robots-dice-and-card-game (started at 7pm Thursday 10/23). The group is hoping to play the game with as many new players as possible all around the country while hoping to hit their goal by December!