Too many posts on demonstrations and work in progress on videos of ARToolkit_MMD (Augmented Reality Toolkit MikuMikuDance), but no instruction manual on how to use it.

demonstration videos, because if a picture is a thousand words, then a video must be a novel
just miku singing secret garden [link]
she now has friends - [link]

VMDSpectrum - [link] MMDAgent - [link]

Description:
ARToolkit is a program used to display models or images on a webcam video by detecting a marker (similar technology: facial recognition). By calculating how the marker image is deformed, the program will know how the marker is oriented in real life. Using that information, it inserts a 3D model with the right orientation into the video using the marker as the center point.
Since MikuMikuDance is a free and powerful 3D modeling software, it can be used to insert a 3d model, motion data, and wave file data stored in the usual .pmm files in augmented reality.


TUTORIAL
1. download MikuMikuDance (mine is directx9 version) from [link]

2. to download ARToolkit_MMD, first go to [link]

3. go to this link [link] Click the obvious button, then on the next screen, click the one that says in english 'download now'

4. extract the files.

5. open the ARToolkit/Data folder, then print out the .pdf named pattSample1 on a piece of paper. for best results, you want to attach it to a hard surface.

6. back in computerland, open the ARToolkit folder, open up the .ini file named setting.

7. change the #MMDDataFolderName to where you extracted your MikuMikuDance Data folder.

8. change the PMMFileName to the path where the .pmm file you want to use is located

9. you can change the scale to get a larger or smaller scaled model.

10. save and exit the .ini file.

11. run ARToolkit_MMD.exe

12. if you have a 64 bit computer, it should be screwing up saying : msvcp71 is missing. If so, copy and paste the msvcp71 and msvcr71 .dll files in the ARToolkit folder into Windows SysWOW64 folder (not system 32). run ARToolkit_MMD.exe to confirm that its fixed.

13. a windows blackscreen appears with a window named 'Property Sheet Properties.

14. Frame rate should be as high as you can, check the flip horizontal if youre using a laptop webcam, uncheck if you somehow have your webcam attached to your head. color space, i dont really care. Output size... 640 x 480 have not been giving me problems. larger than that tends to lagg. The default 640 x 400 somehow messes some stuff up a bit

15. if you run into an error, it means some models arent loading up or you didnt change the path earlier properly. if so, open MikuMikuDance, open up that .pmm file and just delete the models it cant load.

16. if you can run it, a webcam output now appears.

17. bring your printed pattern into full view of the webcam window. The whole pattern should be visible.

18. the program detects a transformation of the pattern 'matrix' to calculate a viewing orientation.

19. the files in the .pmm file should appear, but not move just yet.

20. these buttons on your keyboard have specific functions: a - toggles opacity of the AR model. s - begins/pauses the dance routine. r - resets the dance routine

21. feel free to move the marker to different orientations to get different views of the whole thing.

22. if your .pmm file has a .wav sound file loaded, this version will play the .wav file as well, provided that its title is in a simple form (no jap, no special characters. A pure english title works best.

23. to play other .pmm files, you need to open the .ini file and change the path to where the new .pmm file you want to load is loaded.

extra info - 10/4/2011
if the webcam has trouble sensing the marker because the marker is not appearing very 'black', then open up the setting.ini file, and search for something named MarkerGrayScaleThreshold. Setting it to a high value (255 maximum number) will make your marker detectable even when its not appearing as 'black' as it should be.
if the surroundings are too black, then setting the number to a lower value will help detect the marker in lower light conditions.
Its default setting is 128.

This dance routine is modified from the pre-existing Sample(Allstar).pmm already included in MikuMikuDance. I deleted the two Dummy bone models and the entire stage.

Credits where credits are due
Animasa, m2, crypton future media - the dance routine and all .pmd models used
Project Diva Miku performing the Secret Garden
ARToolkit_MMD's creators at the abovementioned website

some random youtube vids using the same tech
Go my way miku, neru, haku - [link]
caramelldansen - [link]
Interstellar flight with some um.... 'cool features' which i dont know how to do in MMD - [link]

ARToolkit has been known to either NOT work or malfunction with this Rin Append XS mqdl model
[link]

this LAT Rin Append model will work somewhat, but the ribbon IK bones will start going crazy once a dance routine begins. I suspect a physics problem (download via cococity's link)
[link]

full credits and tested models deviation: [link]


update 3/6/2011 I have edited the deviation to show something which will make your augmented reality life a lot easier.
I printed a smaller marker logo and pasted it onto a magnetic bookmark. Now its possible for me to go hands free.
top left: magnetic bookmark attached to shoulder epaulet strap.
middle left, handheld
bottom left, magnetic bookmark attached to sleeve
bottom right, handheld
Also, this works on other surfaces, such as gloves, or drawn on your hand with a permanent marker of sorts. It still needs to be drawn using the proper dimensions and ratios, and must still be straight enough for a proper detection.


update 24/7/2011 Dummy bone will load in ARToolkit_MMD, but some attachments and effects may not load or cause the program to crash. If so, deleting the faulty attachments may work.
update 20/8/2011 Mikumikueffects wont work in ARToolkit.
Also, did a video, and edited the deviation to include one cutscreen from the video. The cutscreen is from the part near the end.
[link]