Year 2018

March

1. Probabilistic Video Generation using Holistic Attribute Control https://arxiv.org/pdf/1803.08085.pdf

? ?a. Videos express highly structured spatio-temporal patterns of visual data. two factors:

? ? ? ? (i) temporally invariant (e.g., person identity), or slowly varying (e.g., activity), attribute-induced appearance, encoding the persistent content of each frame

? ? ? ? (ii) an interframe motion or scene dynamics (e.g., encoding evolution of the person ex- ecuting the action).

? ?b. VideoVAE

? ? ? ?video generation + future prediction.

? ? ? ?generates a video (short clip) by:

? ? ? ? ? ?decoding samples sequentially drawn from a latent space distribution into full video frames.

? ? ? ? ? ? ? -VAE: encoding/decoding frames into/from the latent space

? ? ? ? ? ? ? -RNN: model the dynamics in the latent space. ? ?

? ? ? ? improve the video generation consistency through temporally-conditional sampling and quality

? ? ? ? ? ? ? -structuring the latent space with attribute controls

? ? ? ? ? ? ? -ensuring that attributes can be both inferred and conditioned on during learning/generation

2.Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks

3.Every Smile is Unique: Landmark-Guided Diverse Smile Generation?

Year 2017

-By the Way

?I like this stanford homework paper http://cs231n.stanford.edu/reports/2017/pdfs/323.pdf

1. Dynamics Transfer GAN: Generating Video by Transferring Arbitrary Temporal Dynamics from a Source Video to a Single Target Image

-spatial constructs <---- target image; dynamics <------source video sequence

?To preserve the spatial construct of the target image:

? ? ? ? ? ? ?- the appearance of the source video sequence is suppressed

? ? ? ? ? ? ?- only the dynamics are obtained before being imposed onto the target image. ?(using the proposed appearance suppressed dynamics feature.)

?the spatial and temporal consistencies are verified via two discriminator networks. ?

? ? ? ? ? ? ?- discriminator A validates the fidelity of the generated frames appearance,

? ? ? ? ? ? ?-? B validates the dynamic consistency of the generated video sequence.

Results:

? ? ? ? ? ? ?- successfully transferred arbitrary dynamics of the source video sequence onto a target image

? ? ? ? ? ? ?- maintained the spatial constructs (appearance) of the target image while generating spatially and temporally consistent video sequences.

Note: It is ### everything (Literature Review in its intro) because it is quite new.

2. Deep Video Generation, Prediction and Completion of Human Action Sequences https://arxiv.org/pdf/1711.08682.pdf

3. Video Generation from Text https://arxiv.org/pdf/1710.00421.pdf

-Hybrid VAE plus GAN

-Two parts:

-Static( Using gist to sketch text-conditioned background color and object layout (LSTM, RNN structure)

-Dynamic ( A text2Filter. )

-3.3 Text2Filter

-Note: Quite compact. Need time to digestilter

4. Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks

? ?https://arxiv.org/pdf/1709.07592.pdf

5. MoCoGAN: Decomposing Motion and Content for Video Generation

? ?https://arxiv.org/pdf/1707.04993.pdf

6. To Create What You Tell: Generating Videos from Captions

? ? https://www.microsoft.com/en-us/research/wp-content/uploads/2017/11/BNI02-panA.pdf

-Temporal GANs conditioning on Captions, namely TGANs-C

? ? ?- transformed into a frame sequence with 3D spatio-temporal convolutions.

? ? ? -? GAM evaluation metric ( Section 3.4 Experimental Setting)

- ?Model Architecture

? ? ? ? ? ? -3.1.1 Generator

? ? ? ? ? ? ? ? ? ? ?-Given a sentence ??, a bi-LSTM is utilized to contextually embed the input word sequence, ?+ a LSTM- based encoder to obtain the sentence representation S.?+ concatenated input of the sentence representation S and random noise variable z.synthesize realistic videos with these

? ? ? ? ? ? ?-3.1.2 The discriminator network ?? includes three discriminators:

? ? ? ? ? ? ? ? ? ? ? ? ? ?a.video discriminator classifying realistic videos from generated+ optimizes video-caption matching? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ?b. frame discriminator( between real and fake frames)and aligning frames with the conditioning caption

? ? ? ? ? ? ? ? ? ? ? ? ? ?c. motion discriminator emphasizing that the adjacent frames in the generated videos run smoothly

? ? ? ? ? ? ? -3.1.3 The whole part trained with 3 losses:video-level matching-aware loss, frame-level matching-aware loss and temporal coherence loss

? ? ? ? ? ? ? ? ? ?.

? ?Year 2016

1. Generating Videos with Scene Dynamics

? ?? https://arxiv.org/abs/1609.02612

- a spatio-temporal convolutional architecture

- untangles the scene’s foreground from the background.

- experiments show the model internally learns useful features for recognizing actions with minimal supervision,

- scene dynamics are a promising signal for representation learning.

- Slides : https://pdfs.semanticscholar.org/presentation/7188/6726f0a1b4075a7213499f8f25d7c9fb4143.pdf