Shop Mobile More Submit  Join Login
MMDAgent Tutorial by AddestorionVayanis MMDAgent Tutorial by AddestorionVayanis
In the same vein as the Artoolkit_MMD tutorial [link] , the following is what I currently know about MMDAgent Voice Recognition software developed by the Nagoya Institute of Technology

Other tutorial links (VMDSpectrum [link] )

NOTICE - 5 September 2012
This tutorial may be out of date compared to MMDAgent 1.3 which was released last august. I am having issues changing all my commands to suit the newer version, so i am still using a backdated version. However, the sample.fst and .mdf files do have some pointers on what the slight alterations are.

A speech recognition program developed by the Nagoya Institute of Technology using a variety of open source software, namely the OpenJTalk voice synthesizer, Julius voice recognition and MikuMikudance 3D rendering program.
Nico-nico douga - [link]

This is roughly what it would look like and what you could do with it: (this is after my edits)
summary of things you can do with it:
- have her respond to your vocal commands
- have all your motion datas and wave files on standby
- drag and drop models, even while halfway dancing
- can detect certain keyboard commands to replace vocal commands.

This is how i usually use MMDAgent to proofcheck models that i download from here.

1. Go to [link]
2. Download the MMDAgent binary package for 32 bit windows
3. Download the Sample Script Binary Package
4. Unzip both files

1. The binary package for 32 bit windows has a .exe file named mmdagent.exe
2. The Sample Script Binary Package has a file named MMDAgent_Example.mdf
3. To run it, you need to drag the MMDAgent_Example.mdf and drop it onto the MMDAgent.exe icon. (you can’t run mmdagent.exe first, and then drag and drop the mmdagent_example.mdf onto it. Nothing happens)
4. A window should appear with a model named mei.
5. To make things run faster, you could just right click the .mdf file and set it to always open with mmdagent.exe. Doing so will enable you to just doubleclick the .mdf and it will load up in mmdagent.exe.

1. The default sample script responds to very limited words. This is a list of [command] – (response) - <action> - {additional notes}
a. [konnichiwa] – (konnichiwa) - <bows down to you>
b. [sayonara/bai bai] – (sayonara) - <waves at you>
c. [homupeji] – (MMDAgent no, hōmupēji o hyōji shimasu) - <loads up the MMDagent homepage in your default browser)
d. [Furusukurīn] – (Sukurīn no settei o, henkō shimashita) - <an ‘f’ button keyboard command will be issued to MMDagent, which makes it toggle fullscreen mode>
e. [toshokan] – (gives you directions to the library) - <The menu model disappears and instead loads up a map of the university, highlighting the location of the library. The map disappears once she is done talking and the menu reverts>
f. [anata _ dare] – (self introduction of herself) - <some hand gestures to refer to herself> - {it doesn’t matter what you say in between anata and dare, but both words need to be said in the same sentence}
g. [Arigato] – (youre welcome, speak to me at any time) - <eye movements and slight swaying> - {notice a tone of voice shift between youre welcome and the second sentence. Demonstrates the different expressions available for use}
h. [kawaii/ kirei] – (Either says arigato or Hazukashiidesu using embarrassed tone) - <no special movements> - {demonstrates use of random numbers to vary responses}
2. Keyboard commands are available in the readme file included in the mmdagent32. Notable commands are:
a. lowercase d – toggles the full log. You will be able to see a transcript of what she thought you heard, and any actions she is undertaking.
b. Capital J – toggles the microphone input bar. The blue bar indicates current sound level, the white line indicates the sound detection treshhold and the orange line indicates speech detection. Use your control panel>>sound >>recording>>microphone to control the sensitivity of the microphone. If it is noisy, set it to lower values, but you’ll have to talk louder into the microphone. Adjust it until the ambient noise doesn’t cause the blue bar to exceed the white line.
c. lowercase l (elle, not eye) – toggles head looking at mouse. mei’s head will track your mouse.
d. capital F – toggles event/detection logs. These logs are to the right of mei (your right). You’ll have to rotate the view so you can see it.
e. Other commands are in the readme file.
3. if you drag and drop music files or mikumikudance .vmd files onto the mmdagent, it will play or run the motion. If the music file is in the same folder as a motion data (.vmd) file with the exact same name as the music file, both the motion data and music will play together.

1. The sample script has a file named mmdagent_example.fst.
2. By default it is not associated with a program, but you can open it in notepad.
3. You will be able to see its full programming, with instructions on how to program it, plus all events and actions which can be taken.
4. The codes are arranged neatly so that you will learn how to perform your own basic programming. Note that the symbol # means comment (whatever is written along that same line will not affect the code and is used just as a comment)
a. 0011-0020 – initial setup and loading of models
b. 0021-0030 – teaches you how to use timers and single loop motions
c. 0031-0040 – teaches you how to use simple voice recognition, speech synthesis and motion.
d. 0041-0050 – teaches you how to queue two or more speech sentences together using SYNTH_EVENT_STOP
e. 0051-0060 – teaches you how to use different voice types. They are; normal, angry, bashful, happy and sad (the folder names in MMDAGENT>>Voice)
f. 0061-0070 – teaches you how to use random numbers to varied responses. This is the basis of IF/THEN/ELSE type programming. The random number in question is the command line 23 (VALUE_SET|random|0|100)
g. 0071-0090 – Teaches you how to add, delete, load, add motion to and change motion of models.
h. 0091-0100 – similar to 0031-0040. No need to reexplain myself.
i. 0101-0110 – Teaches you how to run programs. Webpages will load in your default browser. Programs can be loaded if you provide the full path to it (C:folder1subfolder3whatever.exe). Note: there must be NO SPACEBARS in between the path or else it will not load. You could either:
i. delete any spacebars in the path {may cause programs to not work}
ii. change any spacebars to an underscore ( _ ) {may cause programs to not work}
iii. (recommended) create a shortcut to the program you want to run, and place it in a place where the path to the shortcut has no spacebars. The shortcut name must have no spacebars. Shortcuts have an ending file extension .lnk, so you will have to run it like this: C:folder1subfolder3whatever.exe.lnk
j. 0111-0120 – teaches you about issuing keyboard commands. For now, it can issue commands to mmdagent, but if you know whats a windows class type, you can use this program to issue keyboard commands to other programs as well. Mine is able to control my windows media player classic. Will post more info on this subtopic if response to this article is good.
5. You should be able to program a variety of commands

Other Notes:
1. .pmx files will not work
2. Some models which contain certain abnormal textures will also not work.
3. Whether or not mikumikueffect works or not has yet to be determined

1. This girl is hot. Literally. Activating MMDAgent raises the temperature of my intel i5 core to 83 degrees celcius from its usual 60 celcius. The temperature drops almost immediately back to 60 degrees once I shut her off.
2. She’s also high maintenance. She also uses my GeForceGT420M graphics card to an average of 60 % of the GPU clock, making it hard to run any other graphics demanding program at the same time. The clock increases if you activate self shadow (lowercase x keyboard)
3. And she’s a gold digger. A fully charged battery on my laptop can last 90 minutes on average, but only 45 minutes if she’s constantly running (discharge rate of 2.5 Watthour on standard, 5.3 Watthour when she’s running)

I’ld write more, but I figure those who are interested in this will either figure it out eventually, or ask in the comments.

Currently, my programming of my MMDAgent can do the following:
1. Replaces NIT’s Mei with LAT Kagamine Rin Append when starting up
2. Has 20 .pmd models on standby, switchable either by voice command or keyboard key hit.
3. Can play and dance to 18 songs and dance motions on command
4. Loads most of my programs on command.
5. Loads most of my regular webpages on command
6. Loads ALL my regular webpages on a single command.
7. Can stop music and dance when halfway playing, either by voice command or keyboard hit
8. Can change models while dancing
9. Can load up videos and music playlists in media player class, and control Play/Pause, Next or Previous Track, Volume adjustment, and Quitting on command.
10. Can shut herself down on command
11. Can interrupt herself shutting down in case the shutdown command was given unintentionally.
12. And finally, she can counter most Japanese swearing words with more than just 1 or 2 responses.

as per the readme agreement in MMDAgent, the pictures used for this deviation is based on a personal edit of the MMDAgent and should not be confused with the base program which this tutorial is meant to clarify.

Edit 3/10/2011
I changed the file download to a .zip file containing my edited mmdagent_example.fst.
It is my personal edits to see my present code condition, and not the .fst file initially distributed with the sample_script zip file.
To use this file, just copy it over the .fst file in the sample_script folder and use as described above. You are advised to keep your own backup copy of your .fst file just in case.

you may use this for your own entertainment and such. No credits to me are required, although i would be very interested to hear about your own additions, discoveries and innovations that you may develop.

However, I will not be held responsible for any problems it may cause, nor am i obliged to ensure that it works. This is merely to be used as a suggestion on how to code. All other copyrights are in their respective copyright readme files.
<this notification clears the copyright notice requirements in>

[new version uploaded on 18/11/2011 highlighting Value_GET functions and how they are used]

Update 8/10/2011
you can open up the .mdf file in notepad, and there are some initial settings which you can modify at will

notably, the parameter you want to probably reduce is the one named Bullet frames per second (Bullet fps)
The standard 120 frames per second puts a heavy toll on your graphics card. Reducing it down to 30 or so will significantly reduce this load. You're free to set it to any value which suits you.

update 29/11/2012

And because i have been testing way too many models as of late, a new deviation would just about sum up some of the working ones i've been testing. Therefore, some models from pretty serious modelmakers on deviantart are in order...

V3 From middle and clockwise,
~Xenosnake 's Cyber Lat Gear Miku [link] performing Interstellar Flight sm8568000 using a different stage model (credits still being sourced)
MSN-04 Char's Sazabi (sm8550675) performing Packaged (mylist25972080)
~JessHolysAngel 's Electric Love Miku [link] performing Electric Love sm14410247
Nanami's Kisaragi Chihaya (sm12140696) performing Little Match Girl (sm13952228)
~Hanaminasho 's Artistic Miku [link] showing the menu attaching itself to the center of the model regardless of how short it is (shadow problem self repaired)
LAT Heart Beats Miku (sm15355081), LAT Rin Append [link] and ~Mamama7028 schoolgirl project diva Miku showing multiple model functionality [warning: every extra model puts extra strain on your CPU and GPU]
~midnighthinata 's Daughter of Evil Rin [link] performing Romeo and Cinderella (with the event/action log activated)
*capriciously-calm 's Yellow LAT Miku [link] performing Yellow [link] with Bone detection activated
=Jjinomu 's Rolling Girl LAT Miku [link] performing Two Faced Lovers sm15356537 showing how some shiny effects still load in MMDagent, although not sure why some do not.
LAT Rin Append [link] displaying a little menu which i created from the Library menu model included in MMDAgent.

V2 Preview inside the zip file credits:
MSN-04 Char's Sazabi (sm8550675) performing Packaged (mylist25972080)
Lat Rin Append [link] performing Go My Way [link]
Nanami's Kisaragi Chihaya (sm12140696) performing Little Match Girl (sm13952228) [note: voice monitor turned off]
Saboten Sakura Miku [link] performing Disappearance of Hatsune Miku ([link])
Kio Gothic Miku [link] performing Disappearance of Hatsune Miku [side logs activated (shift + f)]
~Xenosnake 's Cyber LAT Miku with Bits [link] showing the menu function i added (menu model edited from the library.pmd model in MMDAgent. [note: all event log turned off ]
LAT Heart Beats Miku (sm15355081), LAT Rin Append and ~Mamama7028 schoolgirl project diva Miku showing multiple model functionality [warning: every extra model puts extra strain on your CPU and GPU]

update 13/11/2011
been browsing nico-nico douga
this guy apparently has also been tweaking MMDAgent, although in japanese.

However, his .fst file gave me one important piece of information: how VALUE_GET function works.

in non-programmerish how to use:
State1 State2 <event> VALUE_GET|variablename
state 2 State3 VALUE_EVAL_GET|variablename|variablevalue3
state 2 State4 VALUE_EVAL_GET|variablename|variablevalue4
state 2 State5 VALUE_EVAL_GET|variablename|variablevalue5
state 2 State6 VALUE_EVAL_GET|variablename|variablevalue6
this replaces the extra long code i used to detect the value of the datas.

in iostream.h C++:
this is the switch x function.

Also, check out the readme file he included.
It has like, super loads of credits and references to models, stages and motion datas across nico-nico douga. Trying them out. lol

update 12/12/2011
.pmx files may not be able to be loaded directly, but convert the .pmx to .pmd files using PMDEditor and it will work okay. Generally.

List of tested models
Before we proceed, there are a list of problems i'ld like to abbreviate, explain and propose solutions to:

Ground Shadow problem (GS)
Description - There is a problem which causes ground and self shadows to not load
Cause - MA Ambient Line value set to less than 1 (no edge lines)
Solution - change the value of MA Ambient Line value to at least 1. This will load up the shadow properly in MMDAgent.
Solution Flaw - Sometimes however, this will reveal unsightly black blotches on the model, and currently i am thinking of how to solve this. If you are a modelmaker, your help would be much appreciated because i dont know crap about whats going on in PMDEditor. Worse to worse, turn as many MA bones as possible to 1. If a tested model is marked with GS, it will be followed by MA bones which should be skipped to avoid black blotches

Skirt Physics (SP)
Description - skirt or other parts have physics which cause it to move upwards.
Cause - The Physics (PH in PMDeditor) setting for the loc y coordinate is too high (about 7-10 is considered high)
Solution - change that value to a lower value (i set it to 3 so that its same as the Lat Miku White model)

Texture Not Loading (TNL)
Descriptions - Textures appear white in MMDagent, but work fine in MikuMikuDance/PMDEditor
Cause - Textures may be non-RGB colored (Indexed color)
Identifying cause - jpeg and BMP files cannot be index color and will always load. PNG and TGA files may be indexed color, so if you have to take a wild guess, start with the PNG and TGA files first.
Solution 1 (if you have a photo editing software) - Convert the texture image back to RGB from whatever it was.
Solution 2 (if you dont have a photo editing software) - Open the texture with Paint and save it as jpeg. Then use PMDEditor to make it use the jpeg file as the texture.

All YM_model type kaito, gakupo and hiyama kyoteru [link] - no problems

All Nakao mikus - works
All Yukiwane (ゆきはね) mikus [link] - works after PMX conversion

All Saboten Miku Models [link] - no problems

All LAT type miku, rin and tetos - (for LAT neru, see nico links below) [miku] [link] [teto] [link] [rin] [link] - the face MA does not have a line ambient value of 1. When you do set it to 1, black patches appear above. There is a way to fix it, which is:
Select only the face MA, select the vertices and mesh, then apply Edit>Outline>Average Apex with a very very small number, followed by Edit>Outline>Average. This however will reduce the maximum framerate that can be achieved by this model in MMDAgent.

Saboten Aoki Lapis - GS

Onto Deviantart modelmakers

~Mamama7028 [link]
Project Diva Miku [link] - no problems whatsoever
Project Diva Miku Chibi - [link] - no problems

~Xenosnake 's Cyber LAT Mikus [link] [link] [link] [link] - no problems whatsoever (may contain Kios, Nakaos, Pirons, CSs)

~SakuneUtau04 's Dancer Miku [link] - no problems whatsoever

SeeU version 3.0 - [link] - Seeu 3.0 and 3.0s will work fine. But the 2.5 version and Nio version - SP
Meltdown Rin take 2 - [link] - no problems
Saihate Miku - [link] - a part of her jacket is permanently pointed upwards. Solution: use the PMDViewer to select and delete the bones attached to her jacket, but dont seem to be a part of the IK bones. (using the bone menu wont work, you have to use your mouse to do this)

*capriciously-calm 's Yellow LAT Miku [link] - no problems

Saihate Hatsune and Zatsune Miku [link] : GS - MA8
Rockish punkish miku - [link] - no problems
Rolling Girl (Diva Extend type) - [link] - no problems
Joker LAT Miku - [link] - GS - MA 4, 6, 12 and 15. You may also want to delete the mesh for MA18 (fangs). Then it loads up without problems

~kurumi13 's Neko Miku v2 [link] - GS - MA MA4,11,16,17,18,19, 23,27,28,29,30,31,32 (most of them are extra facial effects)
Notably, this is among the few models with special features which dont crash MMDAgent (the other being the xenosnake models). Even her tail's physics work. Figuring out why.

~JessHolysAngel 's Electric Love, Melody Line and Sweet Devil mikus [link] - no problems whatsoever

Punk Miku, Rin, Teto [link] - Miku and Teto - no problems. Rin - GS - MA6, 12, 24 and 33. Additionally, rin has a glasses positioning problem.

Kokoro Rin - [link] - no problems
Rock Luka Kio type - [link] - GS

Untouchable miku [link] - Advised to use the version without wing physics. For the model with the wing physics enabled, the wings just fall to oblivion, never to be seen again. The same applies to ARToolkit_MMD
Victorian style Miku, Neru, Teto [link] , [link] , [link] . - no problems
Eager Love Revenge Lat miku [link] , - no problems
sweet lolita LAT miku [link] - no problems
Daughter of Evil and World is Mine Rin [link] [link] - no problems

Artistic Miku [link] - GS and TNL
Snow Miku [link] - crashes
Bride Animasa Miku [link] - no problems at all
Dreamy Theather type Luka [link] - GS

~TwilightAnimeLife 's Tonio model - [link] - crashes

Rolling Girl LAT Miku [link] - no problems in MMDAgent. Shininess effect doesnt load in ARToolkit_MMD
LAT Len [link] - no problems
Meltdown Rin - [link] - TNL
Little Red Riding Miku (limited time)- [link] - TNL

cantarella kaito and miku version 2 [link] - no noticable problems
Kon version 2 [link] - GS
Kimono Luka [link] - GS - change Line Ambient vakue to 1 for all except MA9, 12 and 13

DT Append Miku [link] - crashes. May be a non-standard bone problem. Loads normally in ARToolkit_MMD though.

~Haruhi-chan4ever 's Space Miku [link] - GS

~L3nkun 's SeeU version 1.5 - [link] - GS

Sekiranun Graffiti Miku [link] - her chest appears brighter than every other part of the model for some reason. Fix: change all MA diffusions set at 0.8 to 1. Shirt also has a noticeable texture change. Fix: change the texture for MA5, 6 and 7 from 4,5 and 4.bmp to 6.bmp.
Snow LAT Miku [link] - no problems

Kagamine Len LAT type - [link] - no problems

=kogaan 's LAT Kagamine Len - [link] - no problems

Sega Meltdown Rin - [link] - ground shadow problem requires complicated fix with some tradeoffs, but can work in MMDAgent. Awaiting creator's response.

Disappearance LAT Rin - [link] - GS - MA28, 29 and 45.
Disappearance Miku - [link] - no problems

DT Disappearance Miku ~Hanaminasho collab - [link] - crashes MMDAgent, but will load in ARToolkit_MMD. ARToolkit_MMD reveals no ground shadow, so apply the fix below and it should have shadows even in ARToolkit. Source of crash still being assessed

~Ame-Yuki 's Formal Dress LAT Miku [link] - no problems. This model loads up the shininess effect in both MMDAgent and ARToolkit_MMD

=ZiiiChan 's Rolling Girl Miku [link] - change v_nor.png to RGB and delete association to MA15, then everything loads fine.

~Kanahiko-chan 's Beach PD Len [link] - GS

~YellowDesuCake 's Conflict Miku [link] - no problems

~iiRichie 's Summer Miku [link] - GS. Delete all MAs which use for it to work in ARToolkit_MMD

~SatariaMityo 's Monochrome sky LAT miku [link] - no problems

Kio Gothic Miku [link] (新キオ式ミク ゴス衣装) - no problems

Masisi's Chihaya Kisaragi models - [link] - no problems

Obtained via Niconicodouga or Youtube

sm8550675 - Sazabi model - no problems
sm9965772 - Rockman zero model - no problems
sm13567741 - Piron Miku Maid models - no problems
Project Diva Snow miku - [link] - crashes
sm13824038 - 'Gosick' miku (password required) [officially discontinued. Dont bother asking me for it, i abide strictly by author's rules]
sm11360016 - petite miku - SP & (model with groove bones (Grv) causes it to crash)
sm15355081 - Heart Beats miku - no problems
sm14113395 - LAT type Neru - white, normal and black school uniforms no problem. White school uniform crashes
sm14475754 - Aoi Tori Kisaragi Chihaya - hair physics makes it defy gravity (video merely links to the download account, does not actually show the model in question)
TheAnimeLover40's Standard Rin - [link] - no problems
sm14913870 - Kyun Vampire Girl LAT Miku - PMX model - conversion to pmd required. Works normally once in PMD form.

Current crazy ideas
In theory, the following two projects are possible, though both of them will be extremely long because of the sheer number of keyboard detection and the inability of MMDAgent to do simple math:

Project Diva within MMDAgent - using timers and keyboard inputs, it will be possible to have a rhythm game inside MMDAgent, possibly using new models and motion data. However, detecting the time will be the single hardest part, and it gets longer and tougher depending on how many 'keys' you want to key in for a song. (World is Mine has 383 keys)

Raising Simulation - after going through some of the base ideas behind Idolmaster, it may be possible to program MMDAgent to randomly behave and evolve according to some code. The code would probably use 'emotion' variables, and its up to you to balance out these variables, decide what factors influence them, and work out scenarios on how she'ld respond to all of them. It would be a potentially endless code, even if you dont expect her to retain her personality across your usage of the program.

The ideas mentioned above are not easy one bit. It may be simple in concept, but the implementation will be brutal. If you do choose to pursue such projects, i either recommend you to program it in a simpler programming language like Warcraft Map Editor or C++. If it cant work there, dont even think about trying it with MMDAgent.
Since we now know how VALUE_GET works, it may be a lot simpler and faster compared to what i had to use before

MMDAgent homepage - [link]

The wiki of MMDagent. Note that its in japanese. Google translate it if you have to: [link]

Google translate - [link] MMDAgent specific thread - [link] MMDAgent specific thread - [link]
LAT Kagamine Rin Append Module - [link]

got a youtube video up for starters - [link]
Add a Comment:
SoraJakeBlade Featured By Owner Edited Feb 9, 2017  Student General Artist
hmm strange for some reason im following what your doing but its not opening for me 

even dragging and dropping its not opening at all
NightmareMoooooon Featured By Owner Jan 29, 2017  Hobbyist General Artist
Hello, sorry to bother you! I recently started to use this program and I found your tutorial. It's great, but I still have questions: 1) is it Possible in this program to change the speech of the character, it was not a female voice and a male or a robot? 2) is it Possible to make a character a speech in English but "to ask" is it in Japanese? (Excuse me for bad English)
SoraJakeBlade Featured By Owner Edited Jan 16, 2017  Student General Artist
can this be used with Miku MIku Dance? also where can we get the binary package thingy from? you dont give a list
renisyafitri69 Featured By Owner Nov 18, 2016
Thanks for the information, Very useful
Rayman901 Featured By Owner Jul 5, 2016
I have it on my phone. Hard to figure it out. :(
apolline555 Featured By Owner Dec 7, 2015  Student Digital Artist
Hi ! i would create my commands and i've translate the command in japanese and i've tried this :

# 0121-0130 pantsu

1    121  RECOG_EVENT_STOP|パつ              KEY_POST|MMDAgent|F|OFF
121  122  <eps>                               SYNTH_START|mei|mei_voice_angry|せいとうさくの
122  2    SYNTH_EVENT_STOP|mei                <eps>

i would try to do this funny command but when i've opened mmdagent he hasn't worked :(
could you help me please ?
CJanpiel13 Featured By Owner Nov 17, 2015  Student Artist
Loupyboy Featured By Owner Feb 15, 2015  Hobbyist General Artist
Thanks for the help (it's still the same use in 2014 ^_^ )
Kevin-BS23 Featured By Owner Dec 25, 2014  Hobbyist Digital Artist
Sorry to bother you, I need help. I don't know why but each time I drag MMDAgent_Example.mdf to MMDAgent.exe it won't start. Does this app need programs like Microsoft.NET or something else? And do we need to move the "AppData" folder to C:\Users?

Note : I have tried MMDAgent before and it's working, somehow it won't start now. Mine is version 1.4
AddestorionVayanis Featured By Owner Dec 27, 2014
i tested it using whatever version is active right now (27 December 2014) and its working fine.

I copied and pasted the contents of the MMDagent_example-1.4 into the MMDagent_win32-1.5 folder,
dragged and dropped the MMDagent_example.mdf file onto MMDagent.exe and it works.

If you are using the .fst file attached to this deviation, it should no longer work due to new features implemented.  Please use it only as a reference as to how to program any further .fst files which you are programming.

I would like to update this deviation, but i prefer to research stuff which i havent yet deciphered rather than repair older works.
Kevin-BS23 Featured By Owner Dec 27, 2014  Hobbyist Digital Artist
Wait, I have tested this on my computer (not my laptop) and it's working, but it still won't work on laptop.

Well, thanks for replying. Maybe the problem's on the laptop's performance stuff.
KharanosLogic Featured By Owner Aug 6, 2014
  I hope I'm not bothering you, but I need help. I followed to process very closely, but when I open MMDagent it shows a blue background with the with a bar (with a blue and orange bars on it) and It shows in red (above the bar): (number of frames)fps 4x MSAA. I tried saying some commands and nothing happens.
bluetennant1998 Featured By Owner Nov 20, 2015  Student General Artist
Same happened to me.
AddestorionVayanis Featured By Owner Aug 6, 2014
you did drag and drop MMDagent_example.mdf from their website onto MMDagent.exe right?

If you used the one i provided, it should not work, because it references models in specific locations on my hard drive.  Thus all you should be seeing is a blue screen.

My copy is there so that people can see how i programmed the .fst file by using notepad.  Beyond that, you need to start from the Sample Script binary package from MMDagent's website.
KharanosLogic Featured By Owner Aug 6, 2014
  Thank you!
TheDeluxe16 Featured By Owner Edited Jul 8, 2014
Ello, I'm new here. I tried to use the script you provived in the download link but when I try to open MMDAgent it says disk is missing or something like that, PLEASE HELP ME!!!!!!
AddestorionVayanis Featured By Owner Jul 9, 2014
got a screen capture of what occurred?  just upload it to deviantart pr something.  You can delete it after i see whats the problem.

And you need to describe what you did in detail.
TheDeluxe16 Featured By Owner Jul 10, 2014
Here is the picture of the error and some additional…
AddestorionVayanis Featured By Owner Jul 11, 2014
try doing it with the mmdagent_example files from the mmdagent website, and not the .fst file that i included for download.  My .fst file was only meant to be opened in notepad so people could see the way i programmed her.
It should malfunction because all my models were stored in my drive D, and since you dont have all my models, it will malfunction
TheDeluxe16 Featured By Owner Jul 12, 2014
Ok, I found how to program her. I finally got her to boot up to the LAT Miku Gothic Model and she now knows more words and commands. She can open Dayz and somemore of my applications. I just kept studying your .fst file and did some experimenting and 16 hours later I got it. Thanks bro.
AddestorionVayanis Featured By Owner Jul 15, 2014
there are advanced level MMDagent enthusiasts in the niconico world, but so far, i am unable to understand the way they programmed their MMDagent programs.  Seems you could be able to force it to recognize words outside the inbuilt vocabulary.

But so long as simple gets the job done, just work with that.
MMDKayKay Featured By Owner Jun 1, 2014  Student Digital Artist
I see SNSD... Time to FANGIRL!
lloonnee2 Featured By Owner May 3, 2014
Hi, new user here.

Just a few questions that I couldn't seem to find the answers to:

1. What does <eps> mean, and what does it do?
2. What does RECOG_MODIFY|USERDICT_SET|(dictionary file name) do?
3. Some parts look like Visual Basic's subroutine to me. (Eg: The Hello/other sequence(s) jumps back to the Idle Behaviour subroutine state '2')
1    31   RECOG_EVENT_STOP|こんにちわ         SYNTH_START|mei|mei_voice_normal|こんにちは。
31   32   <eps>                               MOTION_ADD|mei|action|Motion\mei_greeting\mei_greeting.vmd|PART|ONCE
32   2    SYNTH_EVENT_STOP|mei                <eps>

2    21   <eps>                               TIMER_START|idle1|20
21   22   TIMER_EVENT_START|idle1             TIMER_START|idle2|40
22   23   TIMER_EVENT_START|idle2             TIMER_START|idle3|60
23   1    TIMER_EVENT_START|idle3             VALUE_SET|random|0|100

And looking at some old code that came from a reply you had:

1 7000 KEY|i VALUE_GET|modeltype

7000 8001 VALUE_EVENT_GET|modeltype|1 MODEL_CHANGE|mei|Model\Miku\MikuLAT\LATwhite2.pmd
7000 8002 VALUE_EVENT_GET|modeltype|2 MODEL_CHANGE|mei|Model\Miku\KioGothicmiku2\Gothicmiku2.pmd
7000 8003 VALUE_EVENT_GET|modeltype|3 MODEL_CHANGE|mei|Model\Rin\RinAppendLAT\rinappend.pmd

8001 2 <eps> VALUE_SET|modeltype|2
8002 2 <eps> VALUE_SET|modeltype|3
8003 2 <eps> VALUE_SET|modeltype|1
I assume this means that we'll be able to create seperate subroutines for different actions? (Such as different Idle behaviour starting on 3-5, execution of different sets of commands under a particular subroutine.)

4. What would be the command to close the program? It would work quite well to end the program for the Bye sub.
5. Voice banks. Can't the Vocaloid program be used to play pre-recorded responses instead of the HTS system?
6. I'm curious as to why there are two folders for mei in both the Model & Expression folders, titled 'mei' and sd_mei'. What is the sd_mei folder used for?

Thank you for drafting a pretty detailed guide on how to get this program working. Hopefully I can use it to automate some tasks in the future like you do ^^ (though I can't seem to find your edited script to look through, I think it has been removed.)
AddestorionVayanis Featured By Owner May 4, 2014
1. currently, i am unaware of what <eps> stands for, but you should treat it as 'nothing'.  If youre a programmer, i suppose it would be the same as a 'null'

2. The command RECOG_MODIFY|USERDICT_SET|(dictionary file name) does exactly what it says: it loads up another dictionary file.  The inbuilt dictionary files are the ones in the Appsdata\Open_Jtalk\sys.dic.   It is a dictionary file calibrated against japanese newscasters, so thats the language you'll need to speak to use this program.  When you open up MMDAgent_example.fst, it also loads up MMDAgent_example.dic, which provides extra transcribing.  However, i did not devote time to figuring out how this works, so this is as far as i know.

3. Yes, you can pretty much make your own subroutines.  Each line just jumps from different states, and all you need to do is tell it what state its currently at, and what state it needs to jump to after it fulfills its condition and executes its action.  If you combine this all into an elaborate sequence of interlinked actions, your program can pretty much do anything.

4.  The command to close the program is the escape key.  The program can issue a 'key' command to itself using the command KEY_POST.   To shutdown, you can use the command KEY_POST|MMDAgent|ESCAPE|OFF to trigger.

5. I'm not a programmer, so i cant answer this question.  However, if you dont like the default voice, you can tweak it.  Open up MMDAgent_Example.ojt using notepad.  What opens is a document which can control stuff like pitch, speaking rate, gender parameter, etc.  This is used to generate different voice profiles for different emotions.  Changing the values should give you slightly different voices.  Your job to tweak it until you find a voice that you do like.  Mine is more loli-sounding than the default.

6. sd_mei and mei are the folders for different versions of the 'mei' Mikumikudance model, which is in .pmd format.  If youre wondering, sd stands for 'super deformed', and will give you a model with an oversized head, with an undersized body.  This art style is typically used by Good Smile Company's 'Nendoroid' line of cutesy figurines.  You can switch the model MMDagent uses by redirecting the model add function to MODEL_ADD|mei|Model\sd_mei\sd_mei.pmd|0.0,-4.5,0.0|0.0,0.0,0.0| (more or less)

quote: [though I can't seem to find your edited script to look through, I think it has been removed.]
response: My script should not work because all my model, motion and .wav files are missing from the folder.  My MMDagent folder is kind of merged together with my main MMD folder so it can access all the models pretty easily.  I only uploaded it so you can
lloonnee2 Featured By Owner May 5, 2014
Thank you for your prompt and detailed reply. This might get interesting >:D

After further playing around with the program/discussing the example code with a C#/C++ coder friend, here's what I came up with.

1. <esp> appears to refer to a call function within the C# (or C++, I forgot which) language which is short form for Espilon. Generally used for floating double digit numbers to achieve a high accuracy close to zero, it's a rather obscure command/syntax within the C# language, since not many people use it. Essentially like you said, it represents a value almost zero, and it looks like it acts as an 'end' state for the program to execute an action without any input. I suppose it could be useful for programming her to do stuff without any user input/having to set a timer as 0 just to achieve an instantenous effect. A program that takes initiative, finally~

(Also, that ^ might be some form of evidence that this program was made in C# or C++)

2. Intriguing. Perhaps if there is a way to input English into either dictionaries, the program can recognize English input and subsequently either English or Japanese inputs can be used.

3. Yea, awesome. Programming might take some time to make this a fun voice-activated interface, but it should be worth it. After basic parts are done, probably a personality can be created, with a single subroutine being used for complete mood (angry/sad/bored, etc) changes.

4. Hm, okay. Perhaps creating a simple batch file to shut down the computer and running that before telling her to shutdown the program might work for issuing of a simple command to shutdown the computer simply and cleanly.

5. Strange. It seemed like the program directly input voices from a few types of personalities/voice types (bashful, angry, etc) then reads the output text in that manner. I might have missed that piece of code somewhere when browsing through, or maybe I didn't open the ojt file long enough. If this works then, probably it would be sufficiently simple to find something that I like. 

I thought that as an alternative, the voice could could come from, say, an .mp3, .wma or .wav file that has been created with the Vocaloid Synthesizer software. Considering it would be quite hard to mesh the Vocaloid Synthesizer software with whatever programming has been done (unless some magic plug-in can be found to call the Vocaloid software from wherever it's hiding and generate the vocal output), this might be the only realistic chance of getting a Vocaloid voice to be the output. However, it might be a bit clunky having a windows media player opening every single time she replies, so that might not be an ideal solution.

6. Ah I see, thanks.

7. I see, I would have liked to have a look at how you programmed certain functions despite the script not being able to run on my computer, because some commands like the GET_VAR function look potentially powerful and I'd like to see how you manipulated them. In any case, maybe when my script has been completed to a point where it does what I want it to do, I'm opening to sharing it if anyone's interested (Currently using v1.4)

8. Looking at the promotional video, they showed the program running on a phone. Considering it caused a considerable spike on your i5 and certainly ate up a fair bit of processing power and ram on my i5 laptop, I fail to see how they could have ran that program on a tiny phone. Must have been a pretty powerful phone, I think, or some functions of the program that were intensive (like dancing, possibly) were stripped or their visual quality reduced.

Nevertheless, I didn't see anywhere for a port to get the MMDAgent program working on a phone. How do you think they accomplished it? Theoretically if it worked on a phone, a tablet would be able to run that as well (considering it has a bit more processing ability than a smartphone). Add in syncing capabilities across all 3 platforms and we'll have a worthy (and cuter) competitor to Siri, Cortana and Google Now :D
AddestorionVayanis Featured By Owner May 7, 2014
5. the speech output can be changed by using the SYNTH_START|mei|mei_voice_normal|.  If you change the mei_voice_normal to the other types of voice, such as mei_voice_bashful, it will use the parameters you defined for that particular voice.  So it can do stuff like angry, shy, happy, all from a single ojt file.

7. the VALUE_GET|variable is used in the following format:

4884  4885  <eps>                VALUE_GET|themecostume
4885  5905  VALUE_EVENT_GET|themecostume|0    <eps>
4885  4886  VALUE_EVENT_GET|themecostume|1    MODEL_CHANGE|mei|Model\Miku\MikuLATAP\MikuLATAP.pmd

for this to work, all possible states coming out from the VALUE_GET state must pass through a VALUE_EVENT_GET|variable.  This only works for integers, and at least one of the VALUE_EVENT_GET values must be equal to one of the potential values that your variable can hold, or else the program will just stop there.  There is currently no in-game method of getting the value of the variable automatically to be used for further actions, and you need to program the logic of it yourself.

8.  i'm still trying to get it to run on my tab, but no luck still.  The source code isnt immediately available by the nagoya institute.  Its an external application by Carnegie Mellon's School of Computer Science, and has the following link:…
I am unfamiliar with the android application system, and this output does require a certain degree of android knowledge which i am not willing to invest time into deciphering.
lloonnee2 Featured By Owner May 9, 2014
I see.
I'm looking into trying to develop a mood based on Variables that determine the 'mood' of the program, something like that. If only there was a ADD_VALUE|(alias)|(Value) command, it would work great.

Another method of accomplishing this would be to set two lines of code to run simultaneously, like

1      2      <esp>        VALUE_SET|HappyLevel|1
1      2      <esp>        MOTION_ADD|mei|action|Motion\mei_smile\mei_smile.vmd|PART|ONCE

What do you think?

About the Android code, maybe I'll ask if someone can look through it and give me a dumbed down version so that this can be built/compiled for running on Android too.
AddestorionVayanis Featured By Owner May 10, 2014
sadly, if that function existed, my codes could be shortened to a much greater degree than it is now.

Also, two lines of code running simultaneously wont work.  Priority is given to running the highest most line, but the following line runs almost immediately if you run
1      2      <esp>        VALUE_SET|HappyLevel|1
2      3      <esp>        MOTION_ADD|mei|action|Motion\mei_smile\mei_smile.vmd|PART|ONCE

to have the code wait for the previous command to end, you need to use the EVENT_SOMETHING_END conditions.

as for android, you do that.  make your own tutorial and stuff then.  lol
theepiccheesepuff Featured By Owner Apr 26, 2014   Digital Artist
Sorry to bother you again, but since you seem to be the only one who knows how to use this

My program won't respond whenever I say things to it, and I've tried everything....

Is there anyway you could help me figure out how to use it?
AddestorionVayanis Featured By Owner Apr 26, 2014
describe what occurs.

there are a few levels:
1 - the screen is just blue.
2 - there is an MMD model loaded with a bar in the middle without orange lines
3 - there is an MMD model loaded with a bar in the middle, with a continuous orange line
4 - there is an MMD model loaded with a bar in the middle, with orange lines appearing only when you speak

i must know exactly what managed to load before i can answer further questions.
theepiccheesepuff Featured By Owner Apr 27, 2014   Digital Artist
none of those happen, so i think that must have been some sort of glitch...
AddestorionVayanis Featured By Owner Apr 27, 2014
so you dont even have a screen which loads up at all?

not even when you click MMDagent.exe?
theepiccheesepuff Featured By Owner Apr 29, 2014   Digital Artist
well it does, but whenever I talk to the model, it still won't respond...maybe I need to redownload it again...
AddestorionVayanis Featured By Owner Apr 29, 2014
does the orange line only move when you are talking?

If the orange line is always moving even when youre not talking, it means there is a lot of background noise.  Reduce the sound input of your microphone using Control Panel> Sound > Recording > Microphone > Levels

If there is no orange line regardless of how loud you talk, use the same thing as above, but increase the microphone input.

There should only be an orange line while you are talking.  So you must experiment with the settings
theepiccheesepuff Featured By Owner Apr 30, 2014   Digital Artist
ahhh...I'll try that
theepiccheesepuff Featured By Owner Apr 24, 2014   Digital Artist
is there anymore kaito models that work with this?
AddestorionVayanis Featured By Owner Apr 25, 2014
just throw random kaito models onto the default model.  If it works, it should switch the model with the kaito model.

But i dont bother updating the list of tested models.  not worth the effort.
theepiccheesepuff Featured By Owner Apr 25, 2014   Digital Artist
ohhhhh...okey thanks~!
M0rning-0wl Featured By Owner Apr 16, 2014  Hobbyist General Artist
Umm, Hi?
I've done everything I could think of, But MMDAgent doesn't open flat out, Would you happen to know why it might be doing that? Thank you.
AddestorionVayanis Featured By Owner Apr 16, 2014
tried placing the MMDAgent_example folder contents into the MMDagent main folder?
M0rning-0wl Featured By Owner Apr 17, 2014  Hobbyist General Artist
Yeah, I've done that.
This is why I'm so confused.
AddestorionVayanis Featured By Owner Apr 17, 2014
describe what happens after you do the drag and drop of the MMDagent_example.mdf onto the MMDagent.exe program.

The other reason why a direct drag and drop would not work is that the Japanese language isnt enabled on your windows.

Check by the following for windows 8: control panel> languages > japanese
For older versions of windows, it should be under region and language settings.

If japanese isnt enabled, then it should not work.
M0rning-0wl Featured By Owner Apr 17, 2014  Hobbyist General Artist
Well, I've got it open now, and it seems to work but,
When I open it, I saw a blue screen, So I dragged MMDagent_example.mdf onto it expecting it to change or show that it wasn't responding ect.
When I dragged it onto the program, it comes up with the Bar with the blue and orange line .. things,
But the rest of the screen is still blue, am I doing something wrong?
AddestorionVayanis Featured By Owner Apr 17, 2014
you must drag and drop the .mdf file onto the .exe file while in explorer.  If you open mmdagent.exe first, and then drag and drop the .mdf file, it should not work.

The other thing that may be wrong is that the rest of the stuff, like model and voice synthesizer arent in the same folder as the mmdagent_example.

You should be able to press 'd ' even on the blue screen.  If you do, a log menu should pop up.  The next step is to tell me what is on that log screen.
M0rning-0wl Featured By Owner Apr 17, 2014  Hobbyist General Artist
OK, So in the log menu it says something similar to
AddestorionVayanis Featured By Owner Apr 17, 2014
did it by any chance happen to mention:

or model_add could not find model
(1 Reply)
LeeCHEF Featured By Owner Jan 22, 2014
hi there?! on this tutorial, i really wish to know how to add commends but i really do not understand how. would you please help me god ......
AddestorionVayanis Featured By Owner Jan 22, 2014
open up sample_script.fst using notepad.  Then see how the code is arranged.

Use google translate to get the japanese characters you need.

You will also need to have japanese language enabled
Control panel > Language settings > add a language > japanese
LeeCHEF Featured By Owner Jan 22, 2014
hello again! thanks for the response! i have tried what you said, however she does not reconize the pronouciation or perhaps it's just me who programed it wrong, would you happened to have an exemple of how it works?? thx for replying and sorry for the bother
Add a Comment:


Submitted on
September 17, 2011
File Size
2.6 MB


35,738 (6 today)
377 (who?)


Creative Commons License
Some rights reserved. This work is licensed under a
Creative Commons Attribution-Noncommercial 3.0 License.