comparison org/capture-video.org @ 275:da4de661c5d9

I changed Capture Video\!
author Dylan Holmes <ocsenave@gmail.com>
date Wed, 15 Feb 2012 01:15:15 -0600
parents ecafe87ffddc
children 301e91a6c2d1
comparison
equal deleted inserted replaced
272:12e6231eae8e 275:da4de661c5d9
22 video frames. As a result, the game appears to speed up and slow down 22 video frames. As a result, the game appears to speed up and slow down
23 as the computational demands shift, but the end result is perfectly 23 as the computational demands shift, but the end result is perfectly
24 smooth video output at a constant framerate. 24 smooth video output at a constant framerate.
25 25
26 26
27 * Game-time vs. User-time vs. Video-time 27 * Video recording requires a steady framerate
28 28 ** The built-in =Timer= rushes to keep up.
29 A standard JME3 application that extends =SimpleApplication= or 29 #* Game-time vs. User-time vs. Video-time
30 =Application= tries as hard as it can to keep in sync with 30
31 /user-time/. If a ball is rolling at 1 game-mile per game-hour in the 31 Standard JME3 applications use a =Timer= object to manage time
32 game, and you wait for one user-hour as measured by the clock on your 32 in the simulated world. Because most JME3 applications (e.g. games)
33 wall, then the ball should have traveled exactly one game-mile. In 33 are supposed to happen \ldquo{}live\rdquo{}, the built-in =Timer=
34 order to keep sync with the real world, the game throttles its physics 34 requires simulated time to match real time. This means that the
35 engine and graphics display. If the computations involved in running 35 application must rush to finish all of its calculations on
36 the game are too intense, then the game will first skip frames, then 36 schedule: the more complicated the calculations, the more the
37 sacrifice physics accuracy. If there are particuraly demanding 37 application is obligated to rush. And if the workload becomes too
38 computations, then you may only get 1 fps, and the ball may tunnel 38 much to handle on schedule, =Timer= forces the application to cut
39 through the floor or obstacles due to inaccurate physics simulation, 39 corners: it demands fast, approximate answers instead of careful,
40 but after the end of one user-hour, that ball will have traveled one 40 accurate ones. Although this policy sometimes causes physically impossible
41 game-mile. 41 glitches and choppy framerates, it ensures that the user will never be
42 42 kept waiting while the computer stops to make a complicated
43 When we're recording video, we don't care if the game-time syncs with 43 calculation.
44 user-time, but instead whether the time in the recorded video 44
45 (video-time) syncs with user-time. To continue the analogy, if we 45 Now, the built-in =Timer= values speed over accuracy because real-time
46 recorded the ball rolling at 1 game-mile per game-hour and watched the 46 applications require it. On the other hand, if your goal is to record a
47 video later, we would want to see 30 fps video of the ball rolling at 47 glitch-free video, you need a =Timer= that will take its time to
48 1 video-mile per /user-hour/. It doesn't matter how much user-time it 48 ensure that all calculations are accurate, even if they take a long time. In the next section, we
49 took to simulate that hour of game-time to make the high-quality 49 will create a new kind of =Timer=\mdash{}called =IsoTimer=\mdash{}which
50 recording. 50 slows down to let the computer finish all its calculations. The result
51 51 is a perfectly steady framerate and a flawless physical simulation.
52 * COMMENT Two examples to clarify the point: 52
53 ** Recording from a Simple Simulation 53 # are supposed to happen \ldquo live \rdquo, this =Timer= requires the
54 54 # application to update in real-time. In order to keep up with the real world, JME applications cannot
55 *** Without a Special Timer 55 # afford to take too much time on expensive computations. Whenever the
56 # workload becomes too much for the computer to handle on schedule,
57 # =Timer= forces the computer to cut corners, giving fast, approximate
58 # answers instead of careful, accurate ones. Although physical accuracy sometimes
59 # suffers as a result, this policy ensures that the user will never be
60 # kept waiting while the computer stops to make a complicated
61 # calculation.
62
63 #fast answers are more important than accurate ones.
64
65 # A standard JME3 application that extends =SimpleApplication= or
66 # =Application= tries as hard as it can to keep in sync with
67 # /user-time/. If a ball is rolling at 1 game-mile per game-hour in the
68 # game, and you wait for one user-hour as measured by the clock on your
69 # wall, then the ball should have traveled exactly one game-mile. In
70 # order to keep sync with the real world, the game throttles its physics
71 # engine and graphics display. If the computations involved in running
72 # the game are too intense, then the game will first skip frames, then
73 # sacrifice physics accuracy. If there are particuraly demanding
74 # computations, then you may only get 1 fps, and the ball may tunnel
75 # through the floor or obstacles due to inaccurate physics simulation,
76 # but after the end of one user-hour, that ball will have traveled one
77 # game-mile.
78
79 # When we're recording video, we don't care if the game-time syncs with
80 # user-time, but instead whether the time in the recorded video
81 # (video-time) syncs with user-time. To continue the analogy, if we
82 # recorded the ball rolling at 1 game-mile per game-hour and watched the
83 # video later, we would want to see 30 fps video of the ball rolling at
84 # 1 video-mile per /user-hour/. It doesn't matter how much user-time it
85 # took to simulate that hour of game-time to make the high-quality
86 # recording.
87 ** COMMENT Two examples to clarify the point:
88 *** Recording from a Simple Simulation
89
90 **** Without a Special Timer
56 You have a simulation of a ball rolling on an infinite empty plane at 91 You have a simulation of a ball rolling on an infinite empty plane at
57 one game-mile per game-hour, and a really good computer. Normally, 92 one game-mile per game-hour, and a really good computer. Normally,
58 JME3 will throttle the physics engine and graphics display to sync the 93 JME3 will throttle the physics engine and graphics display to sync the
59 game-time with user-time. If it takes one-thousandth of a second 94 game-time with user-time. If it takes one-thousandth of a second
60 user-time to simulate one-sixtieth of a second game time and another 95 user-time to simulate one-sixtieth of a second game time and another
64 user-seconds, then wait, and so on. For every second of user time that 99 user-seconds, then wait, and so on. For every second of user time that
65 passes, one second of game-time passes, and the game will run at 60 100 passes, one second of game-time passes, and the game will run at 60
66 frames per user-second. 101 frames per user-second.
67 102
68 103
69 *** With a Special Timer 104 **** With a Special Timer
70 Then, you change the game's timer so that user-time will be synced to 105 Then, you change the game's timer so that user-time will be synced to
71 video-time. Assume that encoding a single frame takes 0 seconds 106 video-time. Assume that encoding a single frame takes 0 seconds
72 user-time to complete. 107 user-time to complete.
73 108
74 Now, JME3 takes advantage of all available resources. It still takes 109 Now, JME3 takes advantage of all available resources. It still takes
83 When someone watches the video, they will see 60 frames per 118 When someone watches the video, they will see 60 frames per
84 user-second, and $\frac{1}{60}$ video-seconds will pass each frame. It 119 user-second, and $\frac{1}{60}$ video-seconds will pass each frame. It
85 will take exactly one hour user-time (and one hour video-time) for the 120 will take exactly one hour user-time (and one hour video-time) for the
86 ball in the video to travel one video-mile. 121 ball in the video to travel one video-mile.
87 122
88 ** Recording from a Complex Simulation 123 *** Recording from a Complex Simulation
89 124
90 *** Without a Special Timer 125 *** Without a Special Timer
91 You have a simulation of a ball rolling on an infinite empty plane at 126 You have a simulation of a ball rolling on an infinite empty plane at
92 one game-mile per game-hour accompanied by multiple explosions 127 one game-mile per game-hour accompanied by multiple explosions
93 involving thousands of nodes, particle effects, and complicated shadow 128 involving thousands of nodes, particle effects, and complicated shadow
120 will appear to flow two-fifths as fast as user time while the game is 155 will appear to flow two-fifths as fast as user time while the game is
121 running. However, just as in example one, when all is said and done we 156 running. However, just as in example one, when all is said and done we
122 will have an hour long video at 60 fps. 157 will have an hour long video at 60 fps.
123 158
124 159
125 * COMMENT proposed names for the new timer 160 ** COMMENT proposed names for the new timer
126 # METRONOME 161 # METRONOME
127 # IsoTimer 162 # IsoTimer
128 # EvenTimer 163 # EvenTimer
129 # PulseTimer 164 # PulseTimer
130 # FixedTimer 165 # FixedTimer
134 # MetronomeTimer 169 # MetronomeTimer
135 # ConstantTimer 170 # ConstantTimer
136 # SteadyTimer 171 # SteadyTimer
137 172
138 173
139 * =IsoTimer= records time like a metronome 174 ** =IsoTimer= records time like a metronome
140 175
141 The easiest way to achieve this special timing is to create a new 176 The easiest way to achieve this special timing is to create a new
142 timer that always reports the same framerate to JME3 every time it is 177 timer that always reports the same framerate to JME3 every time it is
143 called. 178 called.
144 179
148 183
149 If an Application uses this =IsoTimer= instead of the normal one, we 184 If an Application uses this =IsoTimer= instead of the normal one, we
150 can be sure that every call to =simpleUpdate=, for example, corresponds 185 can be sure that every call to =simpleUpdate=, for example, corresponds
151 to exactly $(\frac{1}{fps})$ seconds of game-time. 186 to exactly $(\frac{1}{fps})$ seconds of game-time.
152 187
153 * Encoding to Video 188 * =VideoRecorder= manages video feeds in JMonkeyEngine.
189
190
191 ** =AbstractVideoRecorder= provides a general framework for managing videos.
154 192
155 Now that the issue of time is solved, we just need a function that 193 Now that the issue of time is solved, we just need a function that
156 writes each frame to a video. We can put this function somewhere 194 writes each frame to a video. We can put this function somewhere
157 where it will be called exactly one per frame. 195 where it will be called exactly once per frame.
158 196
159 The basic functions that a =VideoRecorder= should support are 197 The basic functions that a =VideoRecorder= should support are
160 recording, starting, stopping, and possibly a final finishing step 198 recording, starting, stopping, and possibly a cleanup step
161 there it finilizes the recording (such as writing headers for a video 199 where it finalizes the recording (e.g. by writing headers for a video
162 file). 200 file).
163 201
164 An appropiate interface describing this behaviour could look like 202 An appropiate interface describing this behaviour could look like
165 this: 203 this:
166 204
184 of setup and teardown. 222 of setup and teardown.
185 223
186 =./src/com/aurellem/capture/video/AbstractVideoRecorder.java= 224 =./src/com/aurellem/capture/video/AbstractVideoRecorder.java=
187 #+include ../../jmeCapture/src/com/aurellem/capture/video/AbstractVideoRecorder.java src java 225 #+include ../../jmeCapture/src/com/aurellem/capture/video/AbstractVideoRecorder.java src java
188 226
227
228 ** There are many options for handling video files in Java
229
189 If you want to generate video from Java, a great option is [[http://www.xuggle.com/][Xuggle]]. It 230 If you want to generate video from Java, a great option is [[http://www.xuggle.com/][Xuggle]]. It
190 takes care of everything related to video encoding and decoding and 231 takes care of everything related to video encoding and decoding and
191 runs on Windows, Linux and Mac. Out of all the video frameworks for 232 runs on Windows, Linux and Mac. Out of all the video frameworks for
192 Java I personally like this one the best. 233 Java, I personally like this one the best.
193 234
194 Here is a =VideoRecorder= that uses [[http://www.xuggle.com/][Xuggle]] to write each frame to a 235 Here is a =VideoRecorder= that uses [[http://www.xuggle.com/][Xuggle]] to write each frame to a
195 video file. 236 video file.
196 237
197 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java= 238 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java=
198 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java 239 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java
199 240
200 With this, we are able to record video! 241 With this, we are able to record video!
201 242
202 However, it can be hard to properly install Xuggle. For those of you 243 However, it can be hard to properly install Xuggle. If you would rather not use Xuggle, here is an alternate class that uses
203 who would rather not use Xuggle, here is an alternate class that uses
204 [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner Randelshofer's]] excellent pure Java AVI file writer. 244 [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner Randelshofer's]] excellent pure Java AVI file writer.
205 245
206 =./src/com/aurellem/capture/video/AVIVideoRecorder.java= 246 =./src/com/aurellem/capture/video/AVIVideoRecorder.java=
207 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java 247 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java
208 248
217 257
218 =./src/com/aurellem/capture/video/FileVideoRecorder.java= 258 =./src/com/aurellem/capture/video/FileVideoRecorder.java=
219 #+include ../../jmeCapture/src/com/aurellem/capture/video/FileVideoRecorder.java src java 259 #+include ../../jmeCapture/src/com/aurellem/capture/video/FileVideoRecorder.java src java
220 260
221 261
222 * /Really/ Simple Video Recording 262
223 263
224 The most common case for recording a video is probably to just capture 264 * How to record videos yourself
225 whatever is on your screen exactly as you see it. In this case, this 265
226 method will do. 266 ** Include this code.
267
268 No matter how complicated your application is, it's easy to add
269 support for video output with just a few lines of code.
270 # You can also record from multiple ViewPorts as the above example shows.
271
272 And although you can use =VideoRecorder= to record advanced split-screen videos with multiple views, in the simplest case, you want to capture a single view\mdash{}
273 exactly what's on screen. In this case, the following simple =captureVideo=
274 method will do the job:
227 275
228 #+begin_src java 276 #+begin_src java
229 public static void captureVideo(final Application app, 277 public static void captureVideo(final Application app,
230 final File file) throws IOException{ 278 final File file) throws IOException{
231 final AbstractVideoRecorder videoRecorder; 279 final AbstractVideoRecorder videoRecorder;
252 }; 300 };
253 app.enqueue(thunk); 301 app.enqueue(thunk);
254 } 302 }
255 #+end_src 303 #+end_src
256 304
257 This will select the appropiate backend =VideoRecorder= class 305 This method selects the appropriate =VideoRecorder= class
258 depending on the file name you specify, and insturment your 306 for the file type you specify, and instructs your
259 application to record video to the file. You should still set the 307 application to record video to the file.
260 game's timer to an =IsoTimer= with the desired fps. 308
309
310
311
312 Now that you have a =captureVideo= method, you use it like this:
313
314 - Establish an =Isotimer= and set its framerate :: For example, if
315 you want to record video with a framerate of 30 fps, include
316 the following line of code somewhere in the initializtion of
317 your application:
318 #+begin_src java :exports code
319 this.setTimer(new IsoTimer(30));
320 #+end_src
321
322 - Choose the output file :: If you want to record from the game's
323 main =ViewPort= to a file called =/home/r/record.flv=, then
324 include the following line of code somewhere before you call =app.start()=;
325
326 #+begin_src java :exports code
327 Capture.captureVideo(app, new File("/home/r/record.flv"));
328 #+end_src
329
330
331 ** Simple example
332
261 333
262 This example will record video from the ocean scene from the 334 This example will record video from the ocean scene from the
263 jMonkeyEngine test suite. 335 JMonkeyEngine test suite.
264 #+begin_src java 336 #+begin_src java
265 File video = File.createTempFile("JME-water-video", ".avi"); 337 File video = File.createTempFile("JME-water-video", ".avi");
266 captureVideo(app, video); 338 captureVideo(app, video);
267 app.start(); 339 app.start();
268 System.out.println(video.getCanonicalPath()); 340 System.out.println(video.getCanonicalPath());
271 343
272 I've added support for this under a class called 344 I've added support for this under a class called
273 =com.aurellem.capture.Capture=. You can get it [[http://hg.bortreb.com/jmeCapture/][here]]. 345 =com.aurellem.capture.Capture=. You can get it [[http://hg.bortreb.com/jmeCapture/][here]].
274 346
275 347
276 * Hello Video! 348 ** Hello Video! example
277 349
278 I've taken [[http://code.google.com/p/jmonkeyengine/source/browse/trunk/engine/src/test/jme3test/helloworld/HelloLoop.java][=./jme3/src/test/jme3test/helloworld/HelloLoop.java=]] and 350 I've taken [[http://code.google.com/p/jmonkeyengine/source/browse/trunk/engine/src/test/jme3test/helloworld/HelloLoop.java][=./jme3/src/test/jme3test/helloworld/HelloLoop.java=]] and
279 augmented it with video output as follows: 351 augmented it with video output as follows:
280 352
281 =./src/com/aurellem/capture/examples/HelloVideo.java= 353 =./src/com/aurellem/capture/examples/HelloVideo.java=
308 </iframe> 380 </iframe>
309 381
310 #+END_HTML 382 #+END_HTML
311 383
312 384
313 * Summary 385 * COMMENT More Examples
314 It's quite easy to augment your own application to record video,
315 almost regardless of how complicated the actual application is. You
316 can also record from multiple ViewPorts as the above example shows.
317
318 The process for adding video recording to your application is as
319 follows:
320
321 Assuming you want to record at 30 fps, add:
322
323 #+begin_src java :exports code
324 this.setTimer(new IsoTimer(30));
325 #+end_src
326
327 Somewhere in the initialization of your Application.
328
329 If you want to record from the game's main =ViewPort= to a file called
330 =/home/r/record.flv=, then add:
331
332 #+begin_src java :exports code
333 Capture.captureVideo(app, new File("/home/r/record.flv"));
334 #+end_src
335
336 Before you call =app.start()=;
337
338 * More Examples
339 ** COMMENT Hello Physics 386 ** COMMENT Hello Physics
340 =HelloVideo= is boring. Let's add some video capturing to =HelloPhysics= 387 =HelloVideo= is boring. Let's add some video capturing to =HelloPhysics=
341 and create something fun! 388 and create something fun!
342 389
343 This example is a modified version of =HelloPhysics= that creates four 390 This example is a modified version of =HelloPhysics= that creates four
533 together later. 580 together later.
534 581
535 JME3 Xuggle Aurellem video capture 582 JME3 Xuggle Aurellem video capture
536 583
537 584
538 * Sample Videos 585 * Showcase of recorded videos
539 I encoded most of the original JME3 Hello demos for your viewing 586 I encoded most of the original JME3 Hello demos for your viewing
540 pleasure, all using the =Capture= and =IsoTimer= classes. 587 pleasure, all using the =Capture= and =IsoTimer= classes.
541 588
542 ** HelloTerrain 589 ** HelloTerrain
543 [[http://youtu.be/5_4wyDFwrVQ][HelloTerrain.avi]] 590 [[http://youtu.be/5_4wyDFwrVQ][HelloTerrain.avi]]