Mercurial > cortex
comparison org/capture-video.org @ 275:da4de661c5d9
I changed Capture Video\!
author | Dylan Holmes <ocsenave@gmail.com> |
---|---|
date | Wed, 15 Feb 2012 01:15:15 -0600 |
parents | ecafe87ffddc |
children | 301e91a6c2d1 |
comparison
equal
deleted
inserted
replaced
272:12e6231eae8e | 275:da4de661c5d9 |
---|---|
22 video frames. As a result, the game appears to speed up and slow down | 22 video frames. As a result, the game appears to speed up and slow down |
23 as the computational demands shift, but the end result is perfectly | 23 as the computational demands shift, but the end result is perfectly |
24 smooth video output at a constant framerate. | 24 smooth video output at a constant framerate. |
25 | 25 |
26 | 26 |
27 * Game-time vs. User-time vs. Video-time | 27 * Video recording requires a steady framerate |
28 | 28 ** The built-in =Timer= rushes to keep up. |
29 A standard JME3 application that extends =SimpleApplication= or | 29 #* Game-time vs. User-time vs. Video-time |
30 =Application= tries as hard as it can to keep in sync with | 30 |
31 /user-time/. If a ball is rolling at 1 game-mile per game-hour in the | 31 Standard JME3 applications use a =Timer= object to manage time |
32 game, and you wait for one user-hour as measured by the clock on your | 32 in the simulated world. Because most JME3 applications (e.g. games) |
33 wall, then the ball should have traveled exactly one game-mile. In | 33 are supposed to happen \ldquo{}live\rdquo{}, the built-in =Timer= |
34 order to keep sync with the real world, the game throttles its physics | 34 requires simulated time to match real time. This means that the |
35 engine and graphics display. If the computations involved in running | 35 application must rush to finish all of its calculations on |
36 the game are too intense, then the game will first skip frames, then | 36 schedule: the more complicated the calculations, the more the |
37 sacrifice physics accuracy. If there are particuraly demanding | 37 application is obligated to rush. And if the workload becomes too |
38 computations, then you may only get 1 fps, and the ball may tunnel | 38 much to handle on schedule, =Timer= forces the application to cut |
39 through the floor or obstacles due to inaccurate physics simulation, | 39 corners: it demands fast, approximate answers instead of careful, |
40 but after the end of one user-hour, that ball will have traveled one | 40 accurate ones. Although this policy sometimes causes physically impossible |
41 game-mile. | 41 glitches and choppy framerates, it ensures that the user will never be |
42 | 42 kept waiting while the computer stops to make a complicated |
43 When we're recording video, we don't care if the game-time syncs with | 43 calculation. |
44 user-time, but instead whether the time in the recorded video | 44 |
45 (video-time) syncs with user-time. To continue the analogy, if we | 45 Now, the built-in =Timer= values speed over accuracy because real-time |
46 recorded the ball rolling at 1 game-mile per game-hour and watched the | 46 applications require it. On the other hand, if your goal is to record a |
47 video later, we would want to see 30 fps video of the ball rolling at | 47 glitch-free video, you need a =Timer= that will take its time to |
48 1 video-mile per /user-hour/. It doesn't matter how much user-time it | 48 ensure that all calculations are accurate, even if they take a long time. In the next section, we |
49 took to simulate that hour of game-time to make the high-quality | 49 will create a new kind of =Timer=\mdash{}called =IsoTimer=\mdash{}which |
50 recording. | 50 slows down to let the computer finish all its calculations. The result |
51 | 51 is a perfectly steady framerate and a flawless physical simulation. |
52 * COMMENT Two examples to clarify the point: | 52 |
53 ** Recording from a Simple Simulation | 53 # are supposed to happen \ldquo live \rdquo, this =Timer= requires the |
54 | 54 # application to update in real-time. In order to keep up with the real world, JME applications cannot |
55 *** Without a Special Timer | 55 # afford to take too much time on expensive computations. Whenever the |
56 # workload becomes too much for the computer to handle on schedule, | |
57 # =Timer= forces the computer to cut corners, giving fast, approximate | |
58 # answers instead of careful, accurate ones. Although physical accuracy sometimes | |
59 # suffers as a result, this policy ensures that the user will never be | |
60 # kept waiting while the computer stops to make a complicated | |
61 # calculation. | |
62 | |
63 #fast answers are more important than accurate ones. | |
64 | |
65 # A standard JME3 application that extends =SimpleApplication= or | |
66 # =Application= tries as hard as it can to keep in sync with | |
67 # /user-time/. If a ball is rolling at 1 game-mile per game-hour in the | |
68 # game, and you wait for one user-hour as measured by the clock on your | |
69 # wall, then the ball should have traveled exactly one game-mile. In | |
70 # order to keep sync with the real world, the game throttles its physics | |
71 # engine and graphics display. If the computations involved in running | |
72 # the game are too intense, then the game will first skip frames, then | |
73 # sacrifice physics accuracy. If there are particuraly demanding | |
74 # computations, then you may only get 1 fps, and the ball may tunnel | |
75 # through the floor or obstacles due to inaccurate physics simulation, | |
76 # but after the end of one user-hour, that ball will have traveled one | |
77 # game-mile. | |
78 | |
79 # When we're recording video, we don't care if the game-time syncs with | |
80 # user-time, but instead whether the time in the recorded video | |
81 # (video-time) syncs with user-time. To continue the analogy, if we | |
82 # recorded the ball rolling at 1 game-mile per game-hour and watched the | |
83 # video later, we would want to see 30 fps video of the ball rolling at | |
84 # 1 video-mile per /user-hour/. It doesn't matter how much user-time it | |
85 # took to simulate that hour of game-time to make the high-quality | |
86 # recording. | |
87 ** COMMENT Two examples to clarify the point: | |
88 *** Recording from a Simple Simulation | |
89 | |
90 **** Without a Special Timer | |
56 You have a simulation of a ball rolling on an infinite empty plane at | 91 You have a simulation of a ball rolling on an infinite empty plane at |
57 one game-mile per game-hour, and a really good computer. Normally, | 92 one game-mile per game-hour, and a really good computer. Normally, |
58 JME3 will throttle the physics engine and graphics display to sync the | 93 JME3 will throttle the physics engine and graphics display to sync the |
59 game-time with user-time. If it takes one-thousandth of a second | 94 game-time with user-time. If it takes one-thousandth of a second |
60 user-time to simulate one-sixtieth of a second game time and another | 95 user-time to simulate one-sixtieth of a second game time and another |
64 user-seconds, then wait, and so on. For every second of user time that | 99 user-seconds, then wait, and so on. For every second of user time that |
65 passes, one second of game-time passes, and the game will run at 60 | 100 passes, one second of game-time passes, and the game will run at 60 |
66 frames per user-second. | 101 frames per user-second. |
67 | 102 |
68 | 103 |
69 *** With a Special Timer | 104 **** With a Special Timer |
70 Then, you change the game's timer so that user-time will be synced to | 105 Then, you change the game's timer so that user-time will be synced to |
71 video-time. Assume that encoding a single frame takes 0 seconds | 106 video-time. Assume that encoding a single frame takes 0 seconds |
72 user-time to complete. | 107 user-time to complete. |
73 | 108 |
74 Now, JME3 takes advantage of all available resources. It still takes | 109 Now, JME3 takes advantage of all available resources. It still takes |
83 When someone watches the video, they will see 60 frames per | 118 When someone watches the video, they will see 60 frames per |
84 user-second, and $\frac{1}{60}$ video-seconds will pass each frame. It | 119 user-second, and $\frac{1}{60}$ video-seconds will pass each frame. It |
85 will take exactly one hour user-time (and one hour video-time) for the | 120 will take exactly one hour user-time (and one hour video-time) for the |
86 ball in the video to travel one video-mile. | 121 ball in the video to travel one video-mile. |
87 | 122 |
88 ** Recording from a Complex Simulation | 123 *** Recording from a Complex Simulation |
89 | 124 |
90 *** Without a Special Timer | 125 *** Without a Special Timer |
91 You have a simulation of a ball rolling on an infinite empty plane at | 126 You have a simulation of a ball rolling on an infinite empty plane at |
92 one game-mile per game-hour accompanied by multiple explosions | 127 one game-mile per game-hour accompanied by multiple explosions |
93 involving thousands of nodes, particle effects, and complicated shadow | 128 involving thousands of nodes, particle effects, and complicated shadow |
120 will appear to flow two-fifths as fast as user time while the game is | 155 will appear to flow two-fifths as fast as user time while the game is |
121 running. However, just as in example one, when all is said and done we | 156 running. However, just as in example one, when all is said and done we |
122 will have an hour long video at 60 fps. | 157 will have an hour long video at 60 fps. |
123 | 158 |
124 | 159 |
125 * COMMENT proposed names for the new timer | 160 ** COMMENT proposed names for the new timer |
126 # METRONOME | 161 # METRONOME |
127 # IsoTimer | 162 # IsoTimer |
128 # EvenTimer | 163 # EvenTimer |
129 # PulseTimer | 164 # PulseTimer |
130 # FixedTimer | 165 # FixedTimer |
134 # MetronomeTimer | 169 # MetronomeTimer |
135 # ConstantTimer | 170 # ConstantTimer |
136 # SteadyTimer | 171 # SteadyTimer |
137 | 172 |
138 | 173 |
139 * =IsoTimer= records time like a metronome | 174 ** =IsoTimer= records time like a metronome |
140 | 175 |
141 The easiest way to achieve this special timing is to create a new | 176 The easiest way to achieve this special timing is to create a new |
142 timer that always reports the same framerate to JME3 every time it is | 177 timer that always reports the same framerate to JME3 every time it is |
143 called. | 178 called. |
144 | 179 |
148 | 183 |
149 If an Application uses this =IsoTimer= instead of the normal one, we | 184 If an Application uses this =IsoTimer= instead of the normal one, we |
150 can be sure that every call to =simpleUpdate=, for example, corresponds | 185 can be sure that every call to =simpleUpdate=, for example, corresponds |
151 to exactly $(\frac{1}{fps})$ seconds of game-time. | 186 to exactly $(\frac{1}{fps})$ seconds of game-time. |
152 | 187 |
153 * Encoding to Video | 188 * =VideoRecorder= manages video feeds in JMonkeyEngine. |
189 | |
190 | |
191 ** =AbstractVideoRecorder= provides a general framework for managing videos. | |
154 | 192 |
155 Now that the issue of time is solved, we just need a function that | 193 Now that the issue of time is solved, we just need a function that |
156 writes each frame to a video. We can put this function somewhere | 194 writes each frame to a video. We can put this function somewhere |
157 where it will be called exactly one per frame. | 195 where it will be called exactly once per frame. |
158 | 196 |
159 The basic functions that a =VideoRecorder= should support are | 197 The basic functions that a =VideoRecorder= should support are |
160 recording, starting, stopping, and possibly a final finishing step | 198 recording, starting, stopping, and possibly a cleanup step |
161 there it finilizes the recording (such as writing headers for a video | 199 where it finalizes the recording (e.g. by writing headers for a video |
162 file). | 200 file). |
163 | 201 |
164 An appropiate interface describing this behaviour could look like | 202 An appropiate interface describing this behaviour could look like |
165 this: | 203 this: |
166 | 204 |
184 of setup and teardown. | 222 of setup and teardown. |
185 | 223 |
186 =./src/com/aurellem/capture/video/AbstractVideoRecorder.java= | 224 =./src/com/aurellem/capture/video/AbstractVideoRecorder.java= |
187 #+include ../../jmeCapture/src/com/aurellem/capture/video/AbstractVideoRecorder.java src java | 225 #+include ../../jmeCapture/src/com/aurellem/capture/video/AbstractVideoRecorder.java src java |
188 | 226 |
227 | |
228 ** There are many options for handling video files in Java | |
229 | |
189 If you want to generate video from Java, a great option is [[http://www.xuggle.com/][Xuggle]]. It | 230 If you want to generate video from Java, a great option is [[http://www.xuggle.com/][Xuggle]]. It |
190 takes care of everything related to video encoding and decoding and | 231 takes care of everything related to video encoding and decoding and |
191 runs on Windows, Linux and Mac. Out of all the video frameworks for | 232 runs on Windows, Linux and Mac. Out of all the video frameworks for |
192 Java I personally like this one the best. | 233 Java, I personally like this one the best. |
193 | 234 |
194 Here is a =VideoRecorder= that uses [[http://www.xuggle.com/][Xuggle]] to write each frame to a | 235 Here is a =VideoRecorder= that uses [[http://www.xuggle.com/][Xuggle]] to write each frame to a |
195 video file. | 236 video file. |
196 | 237 |
197 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java= | 238 =./src/com/aurellem/capture/video/XuggleVideoRecorder.java= |
198 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java | 239 #+include ../../jmeCapture/src/com/aurellem/capture/video/XuggleVideoRecorder.java src java |
199 | 240 |
200 With this, we are able to record video! | 241 With this, we are able to record video! |
201 | 242 |
202 However, it can be hard to properly install Xuggle. For those of you | 243 However, it can be hard to properly install Xuggle. If you would rather not use Xuggle, here is an alternate class that uses |
203 who would rather not use Xuggle, here is an alternate class that uses | |
204 [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner Randelshofer's]] excellent pure Java AVI file writer. | 244 [[http://www.randelshofer.ch/blog/2008/08/writing-avi-videos-in-pure-java/][Werner Randelshofer's]] excellent pure Java AVI file writer. |
205 | 245 |
206 =./src/com/aurellem/capture/video/AVIVideoRecorder.java= | 246 =./src/com/aurellem/capture/video/AVIVideoRecorder.java= |
207 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java | 247 #+include ../../jmeCapture/src/com/aurellem/capture/video/AVIVideoRecorder.java src java |
208 | 248 |
217 | 257 |
218 =./src/com/aurellem/capture/video/FileVideoRecorder.java= | 258 =./src/com/aurellem/capture/video/FileVideoRecorder.java= |
219 #+include ../../jmeCapture/src/com/aurellem/capture/video/FileVideoRecorder.java src java | 259 #+include ../../jmeCapture/src/com/aurellem/capture/video/FileVideoRecorder.java src java |
220 | 260 |
221 | 261 |
222 * /Really/ Simple Video Recording | 262 |
223 | 263 |
224 The most common case for recording a video is probably to just capture | 264 * How to record videos yourself |
225 whatever is on your screen exactly as you see it. In this case, this | 265 |
226 method will do. | 266 ** Include this code. |
267 | |
268 No matter how complicated your application is, it's easy to add | |
269 support for video output with just a few lines of code. | |
270 # You can also record from multiple ViewPorts as the above example shows. | |
271 | |
272 And although you can use =VideoRecorder= to record advanced split-screen videos with multiple views, in the simplest case, you want to capture a single view\mdash{} | |
273 exactly what's on screen. In this case, the following simple =captureVideo= | |
274 method will do the job: | |
227 | 275 |
228 #+begin_src java | 276 #+begin_src java |
229 public static void captureVideo(final Application app, | 277 public static void captureVideo(final Application app, |
230 final File file) throws IOException{ | 278 final File file) throws IOException{ |
231 final AbstractVideoRecorder videoRecorder; | 279 final AbstractVideoRecorder videoRecorder; |
252 }; | 300 }; |
253 app.enqueue(thunk); | 301 app.enqueue(thunk); |
254 } | 302 } |
255 #+end_src | 303 #+end_src |
256 | 304 |
257 This will select the appropiate backend =VideoRecorder= class | 305 This method selects the appropriate =VideoRecorder= class |
258 depending on the file name you specify, and insturment your | 306 for the file type you specify, and instructs your |
259 application to record video to the file. You should still set the | 307 application to record video to the file. |
260 game's timer to an =IsoTimer= with the desired fps. | 308 |
309 | |
310 | |
311 | |
312 Now that you have a =captureVideo= method, you use it like this: | |
313 | |
314 - Establish an =Isotimer= and set its framerate :: For example, if | |
315 you want to record video with a framerate of 30 fps, include | |
316 the following line of code somewhere in the initializtion of | |
317 your application: | |
318 #+begin_src java :exports code | |
319 this.setTimer(new IsoTimer(30)); | |
320 #+end_src | |
321 | |
322 - Choose the output file :: If you want to record from the game's | |
323 main =ViewPort= to a file called =/home/r/record.flv=, then | |
324 include the following line of code somewhere before you call =app.start()=; | |
325 | |
326 #+begin_src java :exports code | |
327 Capture.captureVideo(app, new File("/home/r/record.flv")); | |
328 #+end_src | |
329 | |
330 | |
331 ** Simple example | |
332 | |
261 | 333 |
262 This example will record video from the ocean scene from the | 334 This example will record video from the ocean scene from the |
263 jMonkeyEngine test suite. | 335 JMonkeyEngine test suite. |
264 #+begin_src java | 336 #+begin_src java |
265 File video = File.createTempFile("JME-water-video", ".avi"); | 337 File video = File.createTempFile("JME-water-video", ".avi"); |
266 captureVideo(app, video); | 338 captureVideo(app, video); |
267 app.start(); | 339 app.start(); |
268 System.out.println(video.getCanonicalPath()); | 340 System.out.println(video.getCanonicalPath()); |
271 | 343 |
272 I've added support for this under a class called | 344 I've added support for this under a class called |
273 =com.aurellem.capture.Capture=. You can get it [[http://hg.bortreb.com/jmeCapture/][here]]. | 345 =com.aurellem.capture.Capture=. You can get it [[http://hg.bortreb.com/jmeCapture/][here]]. |
274 | 346 |
275 | 347 |
276 * Hello Video! | 348 ** Hello Video! example |
277 | 349 |
278 I've taken [[http://code.google.com/p/jmonkeyengine/source/browse/trunk/engine/src/test/jme3test/helloworld/HelloLoop.java][=./jme3/src/test/jme3test/helloworld/HelloLoop.java=]] and | 350 I've taken [[http://code.google.com/p/jmonkeyengine/source/browse/trunk/engine/src/test/jme3test/helloworld/HelloLoop.java][=./jme3/src/test/jme3test/helloworld/HelloLoop.java=]] and |
279 augmented it with video output as follows: | 351 augmented it with video output as follows: |
280 | 352 |
281 =./src/com/aurellem/capture/examples/HelloVideo.java= | 353 =./src/com/aurellem/capture/examples/HelloVideo.java= |
308 </iframe> | 380 </iframe> |
309 | 381 |
310 #+END_HTML | 382 #+END_HTML |
311 | 383 |
312 | 384 |
313 * Summary | 385 * COMMENT More Examples |
314 It's quite easy to augment your own application to record video, | |
315 almost regardless of how complicated the actual application is. You | |
316 can also record from multiple ViewPorts as the above example shows. | |
317 | |
318 The process for adding video recording to your application is as | |
319 follows: | |
320 | |
321 Assuming you want to record at 30 fps, add: | |
322 | |
323 #+begin_src java :exports code | |
324 this.setTimer(new IsoTimer(30)); | |
325 #+end_src | |
326 | |
327 Somewhere in the initialization of your Application. | |
328 | |
329 If you want to record from the game's main =ViewPort= to a file called | |
330 =/home/r/record.flv=, then add: | |
331 | |
332 #+begin_src java :exports code | |
333 Capture.captureVideo(app, new File("/home/r/record.flv")); | |
334 #+end_src | |
335 | |
336 Before you call =app.start()=; | |
337 | |
338 * More Examples | |
339 ** COMMENT Hello Physics | 386 ** COMMENT Hello Physics |
340 =HelloVideo= is boring. Let's add some video capturing to =HelloPhysics= | 387 =HelloVideo= is boring. Let's add some video capturing to =HelloPhysics= |
341 and create something fun! | 388 and create something fun! |
342 | 389 |
343 This example is a modified version of =HelloPhysics= that creates four | 390 This example is a modified version of =HelloPhysics= that creates four |
533 together later. | 580 together later. |
534 | 581 |
535 JME3 Xuggle Aurellem video capture | 582 JME3 Xuggle Aurellem video capture |
536 | 583 |
537 | 584 |
538 * Sample Videos | 585 * Showcase of recorded videos |
539 I encoded most of the original JME3 Hello demos for your viewing | 586 I encoded most of the original JME3 Hello demos for your viewing |
540 pleasure, all using the =Capture= and =IsoTimer= classes. | 587 pleasure, all using the =Capture= and =IsoTimer= classes. |
541 | 588 |
542 ** HelloTerrain | 589 ** HelloTerrain |
543 [[http://youtu.be/5_4wyDFwrVQ][HelloTerrain.avi]] | 590 [[http://youtu.be/5_4wyDFwrVQ][HelloTerrain.avi]] |