Mercurial > cortex
comparison org/capture-video.org @ 280:301e91a6c2d1
correct formating for capture-video.org
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Wed, 15 Feb 2012 10:31:51 -0700 |
parents | da4de661c5d9 |
children | 7351c9c0c471 |
comparison
equal
deleted
inserted
replaced
279:d8ff1a293b8c | 280:301e91a6c2d1 |
---|---|
10 * The Problem | 10 * The Problem |
11 So you've made your cool new JMonkeyEngine3 game and you want to | 11 So you've made your cool new JMonkeyEngine3 game and you want to |
12 create a demo video to show off your hard work. Screen capturing is | 12 create a demo video to show off your hard work. Screen capturing is |
13 the most straightforward way to do this, but it can slow down your | 13 the most straightforward way to do this, but it can slow down your |
14 game and produce low-quality video as a result. A better way is to | 14 game and produce low-quality video as a result. A better way is to |
15 record a video feed directly from the game while it is | 15 record a video feed directly from the game while it is running. |
16 running. | |
17 | 16 |
18 In this post, I'll explain how you can alter your JMonkeyEngine3 game | 17 In this post, I'll explain how you can alter your JMonkeyEngine3 game |
19 to output video while it is running. The main trick is to alter the | 18 to output video while it is running. The main trick is to alter the |
20 pace of JMonkeyEngine3's in-game time: we allow the engine as much | 19 pace of JMonkeyEngine3's in-game time: we allow the engine as much |
21 time as it needs to compute complicated in-game events and to encode | 20 time as it needs to compute complicated in-game events and to encode |
26 | 25 |
27 * Video recording requires a steady framerate | 26 * Video recording requires a steady framerate |
28 ** The built-in =Timer= rushes to keep up. | 27 ** The built-in =Timer= rushes to keep up. |
29 #* Game-time vs. User-time vs. Video-time | 28 #* Game-time vs. User-time vs. Video-time |
30 | 29 |
31 Standard JME3 applications use a =Timer= object to manage time | 30 Standard JME3 applications use a =Timer= object to manage time in the |
32 in the simulated world. Because most JME3 applications (e.g. games) | 31 simulated world. Because most JME3 applications (e.g. games) are |
33 are supposed to happen \ldquo{}live\rdquo{}, the built-in =Timer= | 32 supposed to happen \ldquo{}live\rdquo{}, the built-in =Timer= requires |
34 requires simulated time to match real time. This means that the | 33 simulated time to match real time. This means that the application |
35 application must rush to finish all of its calculations on | 34 must rush to finish all of its calculations on schedule: the more |
36 schedule: the more complicated the calculations, the more the | 35 complicated the calculations, the more the application is obligated to |
37 application is obligated to rush. And if the workload becomes too | 36 rush. And if the workload becomes too much to handle on schedule, |
38 much to handle on schedule, =Timer= forces the application to cut | 37 =Timer= forces the application to cut corners: it demands fast, |
39 corners: it demands fast, approximate answers instead of careful, | 38 approximate answers instead of careful, accurate ones. Although this |
40 accurate ones. Although this policy sometimes causes physically impossible | 39 policy sometimes causes physically impossible glitches and choppy |
41 glitches and choppy framerates, it ensures that the user will never be | 40 framerates, it ensures that the user will never be kept waiting while |
42 kept waiting while the computer stops to make a complicated | 41 the computer stops to make a complicated calculation. |
43 calculation. | |
44 | 42 |
45 Now, the built-in =Timer= values speed over accuracy because real-time | 43 Now, the built-in =Timer= values speed over accuracy because real-time |
46 applications require it. On the other hand, if your goal is to record a | 44 applications require it. On the other hand, if your goal is to record |
47 glitch-free video, you need a =Timer= that will take its time to | 45 a glitch-free video, you need a =Timer= that will take its time to |
48 ensure that all calculations are accurate, even if they take a long time. In the next section, we | 46 ensure that all calculations are accurate, even if they take a long |
49 will create a new kind of =Timer=\mdash{}called =IsoTimer=\mdash{}which | 47 time. In the next section, we will create a new kind of |
50 slows down to let the computer finish all its calculations. The result | 48 =Timer=\mdash{}called =IsoTimer=\mdash{}which slows down to let the |
51 is a perfectly steady framerate and a flawless physical simulation. | 49 computer finish all its calculations. The result is a perfectly steady |
50 framerate and a flawless physical simulation. | |
52 | 51 |
53 # are supposed to happen \ldquo live \rdquo, this =Timer= requires the | 52 # are supposed to happen \ldquo live \rdquo, this =Timer= requires the |
54 # application to update in real-time. In order to keep up with the real world, JME applications cannot | 53 # application to update in real-time. In order to keep up with the |
55 # afford to take too much time on expensive computations. Whenever the | 54 # real world, JME applications cannot afford to take too much time on |
56 # workload becomes too much for the computer to handle on schedule, | 55 # expensive computations. Whenever the workload becomes too much for |
57 # =Timer= forces the computer to cut corners, giving fast, approximate | 56 # the computer to handle on schedule, =Timer= forces the computer to |
58 # answers instead of careful, accurate ones. Although physical accuracy sometimes | 57 # cut corners, giving fast, approximate answers instead of careful, |
59 # suffers as a result, this policy ensures that the user will never be | 58 # accurate ones. Although physical accuracy sometimes suffers as a |
60 # kept waiting while the computer stops to make a complicated | 59 # result, this policy ensures that the user will never be kept waiting |
61 # calculation. | 60 # while the computer stops to make a complicated calculation. |
62 | 61 |
63 #fast answers are more important than accurate ones. | 62 #fast answers are more important than accurate ones. |
64 | 63 |
65 # A standard JME3 application that extends =SimpleApplication= or | 64 # A standard JME3 application that extends =SimpleApplication= or |
66 # =Application= tries as hard as it can to keep in sync with | 65 # =Application= tries as hard as it can to keep in sync with |
67 # /user-time/. If a ball is rolling at 1 game-mile per game-hour in the | 66 # /user-time/. If a ball is rolling at 1 game-mile per game-hour in |
68 # game, and you wait for one user-hour as measured by the clock on your | 67 # the game, and you wait for one user-hour as measured by the clock on |
69 # wall, then the ball should have traveled exactly one game-mile. In | 68 # your wall, then the ball should have traveled exactly one |
70 # order to keep sync with the real world, the game throttles its physics | 69 # game-mile. In order to keep sync with the real world, the game |
71 # engine and graphics display. If the computations involved in running | 70 # throttles its physics engine and graphics display. If the |
72 # the game are too intense, then the game will first skip frames, then | 71 # computations involved in running the game are too intense, then the |
73 # sacrifice physics accuracy. If there are particuraly demanding | 72 # game will first skip frames, then sacrifice physics accuracy. If |
74 # computations, then you may only get 1 fps, and the ball may tunnel | 73 # there are particuraly demanding computations, then you may only get |
75 # through the floor or obstacles due to inaccurate physics simulation, | 74 # 1 fps, and the ball may tunnel through the floor or obstacles due to |
76 # but after the end of one user-hour, that ball will have traveled one | 75 # inaccurate physics simulation, but after the end of one user-hour, |
77 # game-mile. | 76 # that ball will have traveled one game-mile. |
78 | 77 |
79 # When we're recording video, we don't care if the game-time syncs with | 78 # When we're recording video, we don't care if the game-time syncs |
80 # user-time, but instead whether the time in the recorded video | 79 # with user-time, but instead whether the time in the recorded video |
81 # (video-time) syncs with user-time. To continue the analogy, if we | 80 # (video-time) syncs with user-time. To continue the analogy, if we |
82 # recorded the ball rolling at 1 game-mile per game-hour and watched the | 81 # recorded the ball rolling at 1 game-mile per game-hour and watched |
83 # video later, we would want to see 30 fps video of the ball rolling at | 82 # the video later, we would want to see 30 fps video of the ball |
84 # 1 video-mile per /user-hour/. It doesn't matter how much user-time it | 83 # rolling at 1 video-mile per /user-hour/. It doesn't matter how much |
85 # took to simulate that hour of game-time to make the high-quality | 84 # user-time it took to simulate that hour of game-time to make the |
86 # recording. | 85 # high-quality recording. |
87 ** COMMENT Two examples to clarify the point: | 86 ** COMMENT Two examples to clarify the point: |
88 *** Recording from a Simple Simulation | 87 *** Recording from a Simple Simulation |
89 | 88 |
90 **** Without a Special Timer | 89 **** Without a Special Timer |
91 You have a simulation of a ball rolling on an infinite empty plane at | 90 You have a simulation of a ball rolling on an infinite empty plane at |
149 Now, JME3 will spend $\frac{1}{120}$ of a user-second to step the | 148 Now, JME3 will spend $\frac{1}{120}$ of a user-second to step the |
150 physics tick $\frac{1}{60}$ game-seconds, $\frac{1}{60}$ to draw to | 149 physics tick $\frac{1}{60}$ game-seconds, $\frac{1}{60}$ to draw to |
151 the screen, and an additional $\frac{1}{60}$ to encode the video and | 150 the screen, and an additional $\frac{1}{60}$ to encode the video and |
152 write the frame to disk. This is a total of $\frac{1}{24}$ | 151 write the frame to disk. This is a total of $\frac{1}{24}$ |
153 user-seconds for each $\frac{1}{60}$ game-seconds. It will take | 152 user-seconds for each $\frac{1}{60}$ game-seconds. It will take |
154 $(\frac{60}{24} = 2.5)$ user-hours to record one game-hour and game-time | 153 $(\frac{60}{24} = 2.5)$ user-hours to record one game-hour and |
155 will appear to flow two-fifths as fast as user time while the game is | 154 game-time will appear to flow two-fifths as fast as user time while |
156 running. However, just as in example one, when all is said and done we | 155 the game is running. However, just as in example one, when all is said |
157 will have an hour long video at 60 fps. | 156 and done we will have an hour long video at 60 fps. |
158 | 157 |
159 | 158 |
160 ** COMMENT proposed names for the new timer | 159 ** COMMENT proposed names for the new timer |
161 # METRONOME | 160 # METRONOME |
162 # IsoTimer | 161 # IsoTimer |
180 | 179 |
181 =./src/com/aurellem/capture/IsoTimer.java= | 180 =./src/com/aurellem/capture/IsoTimer.java= |
182 #+include ../../jmeCapture/src/com/aurellem/capture/IsoTimer.java src java | 181 #+include ../../jmeCapture/src/com/aurellem/capture/IsoTimer.java src java |
183 | 182 |
184 If an Application uses this =IsoTimer= instead of the normal one, we | 183 If an Application uses this =IsoTimer= instead of the normal one, we |
185 can be sure that every call to =simpleUpdate=, for example, corresponds | 184 can be sure that every call to =simpleUpdate=, for example, |
186 to exactly $(\frac{1}{fps})$ seconds of game-time. | 185 corresponds to exactly $(\frac{1}{fps})$ seconds of game-time. |
187 | 186 |
188 * =VideoRecorder= manages video feeds in JMonkeyEngine. | 187 * =VideoRecorder= manages video feeds in JMonkeyEngine. |
189 | 188 |
190 | 189 |
191 ** =AbstractVideoRecorder= provides a general framework for managing videos. | 190 ** =AbstractVideoRecorder= provides a general framework for managing videos. |
238 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java= | 237 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java= |
239 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java | 238 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java |
240 | 239 |
241 With this, we are able to record video! | 240 With this, we are able to record video! |
242 | 241 |
243 However, it can be hard to properly install Xuggle. If you would rather not use Xuggle, here is an alternate class that uses | 242 However, it can be hard to properly install Xuggle. If you would |
244 [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner Randelshofer's]] excellent pure Java AVI file writer. | 243 rather not use Xuggle, here is an alternate class that uses [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner |
244 Randelshofer's]] excellent pure Java AVI file writer. | |
245 | 245 |
246 =./src/com/aurellem/capture/video/AVIVideoRecorder.java= | 246 =./src/com/aurellem/capture/video/AVIVideoRecorder.java= |
247 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java | 247 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java |
248 | 248 |
249 This =AVIVideoRecorder= is more limited than the | 249 This =AVIVideoRecorder= is more limited than the |
263 | 263 |
264 * How to record videos yourself | 264 * How to record videos yourself |
265 | 265 |
266 ** Include this code. | 266 ** Include this code. |
267 | 267 |
268 No matter how complicated your application is, it's easy to add | 268 No matter how complicated your application is, it's easy to add |
269 support for video output with just a few lines of code. | 269 support for video output with just a few lines of code. |
270 # You can also record from multiple ViewPorts as the above example shows. | 270 # You can also record from multiple ViewPorts as the above example shows. |
271 | 271 |
272 And although you can use =VideoRecorder= to record advanced split-screen videos with multiple views, in the simplest case, you want to capture a single view\mdash{} | 272 And although you can use =VideoRecorder= to record advanced |
273 exactly what's on screen. In this case, the following simple =captureVideo= | 273 split-screen videos with multiple views, in the simplest case, you |
274 method will do the job: | 274 want to capture a single view\mdash{} exactly what's on screen. In |
275 this case, the following simple =captureVideo= method will do the job: | |
275 | 276 |
276 #+begin_src java | 277 #+begin_src java |
277 public static void captureVideo(final Application app, | 278 public static void captureVideo(final Application app, |
278 final File file) throws IOException{ | 279 final File file) throws IOException{ |
279 final AbstractVideoRecorder videoRecorder; | 280 final AbstractVideoRecorder videoRecorder; |
300 }; | 301 }; |
301 app.enqueue(thunk); | 302 app.enqueue(thunk); |
302 } | 303 } |
303 #+end_src | 304 #+end_src |
304 | 305 |
305 This method selects the appropriate =VideoRecorder= class | 306 This method selects the appropriate =VideoRecorder= class for the file |
306 for the file type you specify, and instructs your | 307 type you specify, and instructs your application to record video to |
307 application to record video to the file. | 308 the file. |
308 | |
309 | |
310 | |
311 | 309 |
312 Now that you have a =captureVideo= method, you use it like this: | 310 Now that you have a =captureVideo= method, you use it like this: |
313 | 311 |
314 - Establish an =Isotimer= and set its framerate :: For example, if | 312 - Establish an =Isotimer= and set its framerate :: For example, if |
315 you want to record video with a framerate of 30 fps, include | 313 you want to record video with a framerate of 30 fps, include |
319 this.setTimer(new IsoTimer(30)); | 317 this.setTimer(new IsoTimer(30)); |
320 #+end_src | 318 #+end_src |
321 | 319 |
322 - Choose the output file :: If you want to record from the game's | 320 - Choose the output file :: If you want to record from the game's |
323 main =ViewPort= to a file called =/home/r/record.flv=, then | 321 main =ViewPort= to a file called =/home/r/record.flv=, then |
324 include the following line of code somewhere before you call =app.start()=; | 322 include the following line of code somewhere before you call |
323 =app.start()=; | |
325 | 324 |
326 #+begin_src java :exports code | 325 #+begin_src java :exports code |
327 Capture.captureVideo(app, new File("/home/r/record.flv")); | 326 Capture.captureVideo(app, new File("/home/r/record.flv")); |
328 #+end_src | 327 #+end_src |
329 | 328 |
382 #+END_HTML | 381 #+END_HTML |
383 | 382 |
384 | 383 |
385 * COMMENT More Examples | 384 * COMMENT More Examples |
386 ** COMMENT Hello Physics | 385 ** COMMENT Hello Physics |
387 =HelloVideo= is boring. Let's add some video capturing to =HelloPhysics= | 386 |
388 and create something fun! | 387 =HelloVideo= is boring. Let's add some video capturing to |
388 =HelloPhysics= and create something fun! | |
389 | 389 |
390 This example is a modified version of =HelloPhysics= that creates four | 390 This example is a modified version of =HelloPhysics= that creates four |
391 simultaneous views of the same scene of cannonballs careening into a | 391 simultaneous views of the same scene of cannonballs careening into a |
392 brick wall. | 392 brick wall. |
393 | 393 |
577 demo application using Xuggle (www.xuggle.com/). Everything is | 577 demo application using Xuggle (www.xuggle.com/). Everything is |
578 explained at http://aurellem.org/cortex/capture-video.html. Here, | 578 explained at http://aurellem.org/cortex/capture-video.html. Here, |
579 four points of view are simultaneously recorded and then glued | 579 four points of view are simultaneously recorded and then glued |
580 together later. | 580 together later. |
581 | 581 |
582 JME3 Xuggle Aurellem video capture | 582 JME3 Xuggle Aurellem video capture |
583 | 583 |
584 | 584 |
585 * Showcase of recorded videos | 585 * Showcase of recorded videos |
586 I encoded most of the original JME3 Hello demos for your viewing | 586 I encoded most of the original JME3 Hello demos for your viewing |
587 pleasure, all using the =Capture= and =IsoTimer= classes. | 587 pleasure, all using the =Capture= and =IsoTimer= classes. |