comparison org/capture-video.org @ 280:301e91a6c2d1

correct formating for capture-video.org
author Robert McIntyre <rlm@mit.edu>
date Wed, 15 Feb 2012 10:31:51 -0700
parents da4de661c5d9
children 7351c9c0c471
comparison
equal deleted inserted replaced
279:d8ff1a293b8c 280:301e91a6c2d1
10 * The Problem 10 * The Problem
11 So you've made your cool new JMonkeyEngine3 game and you want to 11 So you've made your cool new JMonkeyEngine3 game and you want to
12 create a demo video to show off your hard work. Screen capturing is 12 create a demo video to show off your hard work. Screen capturing is
13 the most straightforward way to do this, but it can slow down your 13 the most straightforward way to do this, but it can slow down your
14 game and produce low-quality video as a result. A better way is to 14 game and produce low-quality video as a result. A better way is to
15 record a video feed directly from the game while it is 15 record a video feed directly from the game while it is running.
16 running.
17 16
18 In this post, I'll explain how you can alter your JMonkeyEngine3 game 17 In this post, I'll explain how you can alter your JMonkeyEngine3 game
19 to output video while it is running. The main trick is to alter the 18 to output video while it is running. The main trick is to alter the
20 pace of JMonkeyEngine3's in-game time: we allow the engine as much 19 pace of JMonkeyEngine3's in-game time: we allow the engine as much
21 time as it needs to compute complicated in-game events and to encode 20 time as it needs to compute complicated in-game events and to encode
26 25
27 * Video recording requires a steady framerate 26 * Video recording requires a steady framerate
28 ** The built-in =Timer= rushes to keep up. 27 ** The built-in =Timer= rushes to keep up.
29 #* Game-time vs. User-time vs. Video-time 28 #* Game-time vs. User-time vs. Video-time
30 29
31 Standard JME3 applications use a =Timer= object to manage time 30 Standard JME3 applications use a =Timer= object to manage time in the
32 in the simulated world. Because most JME3 applications (e.g. games) 31 simulated world. Because most JME3 applications (e.g. games) are
33 are supposed to happen \ldquo{}live\rdquo{}, the built-in =Timer= 32 supposed to happen \ldquo{}live\rdquo{}, the built-in =Timer= requires
34 requires simulated time to match real time. This means that the 33 simulated time to match real time. This means that the application
35 application must rush to finish all of its calculations on 34 must rush to finish all of its calculations on schedule: the more
36 schedule: the more complicated the calculations, the more the 35 complicated the calculations, the more the application is obligated to
37 application is obligated to rush. And if the workload becomes too 36 rush. And if the workload becomes too much to handle on schedule,
38 much to handle on schedule, =Timer= forces the application to cut 37 =Timer= forces the application to cut corners: it demands fast,
39 corners: it demands fast, approximate answers instead of careful, 38 approximate answers instead of careful, accurate ones. Although this
40 accurate ones. Although this policy sometimes causes physically impossible 39 policy sometimes causes physically impossible glitches and choppy
41 glitches and choppy framerates, it ensures that the user will never be 40 framerates, it ensures that the user will never be kept waiting while
42 kept waiting while the computer stops to make a complicated 41 the computer stops to make a complicated calculation.
43 calculation.
44 42
45 Now, the built-in =Timer= values speed over accuracy because real-time 43 Now, the built-in =Timer= values speed over accuracy because real-time
46 applications require it. On the other hand, if your goal is to record a 44 applications require it. On the other hand, if your goal is to record
47 glitch-free video, you need a =Timer= that will take its time to 45 a glitch-free video, you need a =Timer= that will take its time to
48 ensure that all calculations are accurate, even if they take a long time. In the next section, we 46 ensure that all calculations are accurate, even if they take a long
49 will create a new kind of =Timer=\mdash{}called =IsoTimer=\mdash{}which 47 time. In the next section, we will create a new kind of
50 slows down to let the computer finish all its calculations. The result 48 =Timer=\mdash{}called =IsoTimer=\mdash{}which slows down to let the
51 is a perfectly steady framerate and a flawless physical simulation. 49 computer finish all its calculations. The result is a perfectly steady
50 framerate and a flawless physical simulation.
52 51
53 # are supposed to happen \ldquo live \rdquo, this =Timer= requires the 52 # are supposed to happen \ldquo live \rdquo, this =Timer= requires the
54 # application to update in real-time. In order to keep up with the real world, JME applications cannot 53 # application to update in real-time. In order to keep up with the
55 # afford to take too much time on expensive computations. Whenever the 54 # real world, JME applications cannot afford to take too much time on
56 # workload becomes too much for the computer to handle on schedule, 55 # expensive computations. Whenever the workload becomes too much for
57 # =Timer= forces the computer to cut corners, giving fast, approximate 56 # the computer to handle on schedule, =Timer= forces the computer to
58 # answers instead of careful, accurate ones. Although physical accuracy sometimes 57 # cut corners, giving fast, approximate answers instead of careful,
59 # suffers as a result, this policy ensures that the user will never be 58 # accurate ones. Although physical accuracy sometimes suffers as a
60 # kept waiting while the computer stops to make a complicated 59 # result, this policy ensures that the user will never be kept waiting
61 # calculation. 60 # while the computer stops to make a complicated calculation.
62 61
63 #fast answers are more important than accurate ones. 62 #fast answers are more important than accurate ones.
64 63
65 # A standard JME3 application that extends =SimpleApplication= or 64 # A standard JME3 application that extends =SimpleApplication= or
66 # =Application= tries as hard as it can to keep in sync with 65 # =Application= tries as hard as it can to keep in sync with
67 # /user-time/. If a ball is rolling at 1 game-mile per game-hour in the 66 # /user-time/. If a ball is rolling at 1 game-mile per game-hour in
68 # game, and you wait for one user-hour as measured by the clock on your 67 # the game, and you wait for one user-hour as measured by the clock on
69 # wall, then the ball should have traveled exactly one game-mile. In 68 # your wall, then the ball should have traveled exactly one
70 # order to keep sync with the real world, the game throttles its physics 69 # game-mile. In order to keep sync with the real world, the game
71 # engine and graphics display. If the computations involved in running 70 # throttles its physics engine and graphics display. If the
72 # the game are too intense, then the game will first skip frames, then 71 # computations involved in running the game are too intense, then the
73 # sacrifice physics accuracy. If there are particuraly demanding 72 # game will first skip frames, then sacrifice physics accuracy. If
74 # computations, then you may only get 1 fps, and the ball may tunnel 73 # there are particuraly demanding computations, then you may only get
75 # through the floor or obstacles due to inaccurate physics simulation, 74 # 1 fps, and the ball may tunnel through the floor or obstacles due to
76 # but after the end of one user-hour, that ball will have traveled one 75 # inaccurate physics simulation, but after the end of one user-hour,
77 # game-mile. 76 # that ball will have traveled one game-mile.
78 77
79 # When we're recording video, we don't care if the game-time syncs with 78 # When we're recording video, we don't care if the game-time syncs
80 # user-time, but instead whether the time in the recorded video 79 # with user-time, but instead whether the time in the recorded video
81 # (video-time) syncs with user-time. To continue the analogy, if we 80 # (video-time) syncs with user-time. To continue the analogy, if we
82 # recorded the ball rolling at 1 game-mile per game-hour and watched the 81 # recorded the ball rolling at 1 game-mile per game-hour and watched
83 # video later, we would want to see 30 fps video of the ball rolling at 82 # the video later, we would want to see 30 fps video of the ball
84 # 1 video-mile per /user-hour/. It doesn't matter how much user-time it 83 # rolling at 1 video-mile per /user-hour/. It doesn't matter how much
85 # took to simulate that hour of game-time to make the high-quality 84 # user-time it took to simulate that hour of game-time to make the
86 # recording. 85 # high-quality recording.
87 ** COMMENT Two examples to clarify the point: 86 ** COMMENT Two examples to clarify the point:
88 *** Recording from a Simple Simulation 87 *** Recording from a Simple Simulation
89 88
90 **** Without a Special Timer 89 **** Without a Special Timer
91 You have a simulation of a ball rolling on an infinite empty plane at 90 You have a simulation of a ball rolling on an infinite empty plane at
149 Now, JME3 will spend $\frac{1}{120}$ of a user-second to step the 148 Now, JME3 will spend $\frac{1}{120}$ of a user-second to step the
150 physics tick $\frac{1}{60}$ game-seconds, $\frac{1}{60}$ to draw to 149 physics tick $\frac{1}{60}$ game-seconds, $\frac{1}{60}$ to draw to
151 the screen, and an additional $\frac{1}{60}$ to encode the video and 150 the screen, and an additional $\frac{1}{60}$ to encode the video and
152 write the frame to disk. This is a total of $\frac{1}{24}$ 151 write the frame to disk. This is a total of $\frac{1}{24}$
153 user-seconds for each $\frac{1}{60}$ game-seconds. It will take 152 user-seconds for each $\frac{1}{60}$ game-seconds. It will take
154 $(\frac{60}{24} = 2.5)$ user-hours to record one game-hour and game-time 153 $(\frac{60}{24} = 2.5)$ user-hours to record one game-hour and
155 will appear to flow two-fifths as fast as user time while the game is 154 game-time will appear to flow two-fifths as fast as user time while
156 running. However, just as in example one, when all is said and done we 155 the game is running. However, just as in example one, when all is said
157 will have an hour long video at 60 fps. 156 and done we will have an hour long video at 60 fps.
158 157
159 158
160 ** COMMENT proposed names for the new timer 159 ** COMMENT proposed names for the new timer
161 # METRONOME 160 # METRONOME
162 # IsoTimer 161 # IsoTimer
180 179
181 =./src/com/aurellem/capture/IsoTimer.java= 180 =./src/com/aurellem/capture/IsoTimer.java=
182 #+include ../../jmeCapture/src/com/aurellem/capture/IsoTimer.java src java 181 #+include ../../jmeCapture/src/com/aurellem/capture/IsoTimer.java src java
183 182
184 If an Application uses this =IsoTimer= instead of the normal one, we 183 If an Application uses this =IsoTimer= instead of the normal one, we
185 can be sure that every call to =simpleUpdate=, for example, corresponds 184 can be sure that every call to =simpleUpdate=, for example,
186 to exactly $(\frac{1}{fps})$ seconds of game-time. 185 corresponds to exactly $(\frac{1}{fps})$ seconds of game-time.
187 186
188 * =VideoRecorder= manages video feeds in JMonkeyEngine. 187 * =VideoRecorder= manages video feeds in JMonkeyEngine.
189 188
190 189
191 ** =AbstractVideoRecorder= provides a general framework for managing videos. 190 ** =AbstractVideoRecorder= provides a general framework for managing videos.
238 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java= 237 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java=
239 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java 238 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java
240 239
241 With this, we are able to record video! 240 With this, we are able to record video!
242 241
243 However, it can be hard to properly install Xuggle. If you would rather not use Xuggle, here is an alternate class that uses 242 However, it can be hard to properly install Xuggle. If you would
244 [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner Randelshofer's]] excellent pure Java AVI file writer. 243 rather not use Xuggle, here is an alternate class that uses [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner
244 Randelshofer's]] excellent pure Java AVI file writer.
245 245
246 =./src/com/aurellem/capture/video/AVIVideoRecorder.java= 246 =./src/com/aurellem/capture/video/AVIVideoRecorder.java=
247 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java 247 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java
248 248
249 This =AVIVideoRecorder= is more limited than the 249 This =AVIVideoRecorder= is more limited than the
263 263
264 * How to record videos yourself 264 * How to record videos yourself
265 265
266 ** Include this code. 266 ** Include this code.
267 267
268 No matter how complicated your application is, it's easy to add 268 No matter how complicated your application is, it's easy to add
269 support for video output with just a few lines of code. 269 support for video output with just a few lines of code.
270 # You can also record from multiple ViewPorts as the above example shows. 270 # You can also record from multiple ViewPorts as the above example shows.
271 271
272 And although you can use =VideoRecorder= to record advanced split-screen videos with multiple views, in the simplest case, you want to capture a single view\mdash{} 272 And although you can use =VideoRecorder= to record advanced
273 exactly what's on screen. In this case, the following simple =captureVideo= 273 split-screen videos with multiple views, in the simplest case, you
274 method will do the job: 274 want to capture a single view\mdash{} exactly what's on screen. In
275 this case, the following simple =captureVideo= method will do the job:
275 276
276 #+begin_src java 277 #+begin_src java
277 public static void captureVideo(final Application app, 278 public static void captureVideo(final Application app,
278 final File file) throws IOException{ 279 final File file) throws IOException{
279 final AbstractVideoRecorder videoRecorder; 280 final AbstractVideoRecorder videoRecorder;
300 }; 301 };
301 app.enqueue(thunk); 302 app.enqueue(thunk);
302 } 303 }
303 #+end_src 304 #+end_src
304 305
305 This method selects the appropriate =VideoRecorder= class 306 This method selects the appropriate =VideoRecorder= class for the file
306 for the file type you specify, and instructs your 307 type you specify, and instructs your application to record video to
307 application to record video to the file. 308 the file.
308
309
310
311 309
312 Now that you have a =captureVideo= method, you use it like this: 310 Now that you have a =captureVideo= method, you use it like this:
313 311
314 - Establish an =Isotimer= and set its framerate :: For example, if 312 - Establish an =Isotimer= and set its framerate :: For example, if
315 you want to record video with a framerate of 30 fps, include 313 you want to record video with a framerate of 30 fps, include
319 this.setTimer(new IsoTimer(30)); 317 this.setTimer(new IsoTimer(30));
320 #+end_src 318 #+end_src
321 319
322 - Choose the output file :: If you want to record from the game's 320 - Choose the output file :: If you want to record from the game's
323 main =ViewPort= to a file called =/home/r/record.flv=, then 321 main =ViewPort= to a file called =/home/r/record.flv=, then
324 include the following line of code somewhere before you call =app.start()=; 322 include the following line of code somewhere before you call
323 =app.start()=;
325 324
326 #+begin_src java :exports code 325 #+begin_src java :exports code
327 Capture.captureVideo(app, new File("/home/r/record.flv")); 326 Capture.captureVideo(app, new File("/home/r/record.flv"));
328 #+end_src 327 #+end_src
329 328
382 #+END_HTML 381 #+END_HTML
383 382
384 383
385 * COMMENT More Examples 384 * COMMENT More Examples
386 ** COMMENT Hello Physics 385 ** COMMENT Hello Physics
387 =HelloVideo= is boring. Let's add some video capturing to =HelloPhysics= 386
388 and create something fun! 387 =HelloVideo= is boring. Let's add some video capturing to
388 =HelloPhysics= and create something fun!
389 389
390 This example is a modified version of =HelloPhysics= that creates four 390 This example is a modified version of =HelloPhysics= that creates four
391 simultaneous views of the same scene of cannonballs careening into a 391 simultaneous views of the same scene of cannonballs careening into a
392 brick wall. 392 brick wall.
393 393
577 demo application using Xuggle (www.xuggle.com/). Everything is 577 demo application using Xuggle (www.xuggle.com/). Everything is
578 explained at http://aurellem.org/cortex/capture-video.html. Here, 578 explained at http://aurellem.org/cortex/capture-video.html. Here,
579 four points of view are simultaneously recorded and then glued 579 four points of view are simultaneously recorded and then glued
580 together later. 580 together later.
581 581
582 JME3 Xuggle Aurellem video capture 582 JME3 Xuggle Aurellem video capture
583 583
584 584
585 * Showcase of recorded videos 585 * Showcase of recorded videos
586 I encoded most of the original JME3 Hello demos for your viewing 586 I encoded most of the original JME3 Hello demos for your viewing
587 pleasure, all using the =Capture= and =IsoTimer= classes. 587 pleasure, all using the =Capture= and =IsoTimer= classes.