Skip to content

Instantly share code, notes, and snippets.

@nateraw

nateraw/rit-demo-stable_diffusion_videos.ipynb

Last active April 7, 2023 19:40
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save nateraw/9ac46a55825611f4a2c8e8b97df8febb to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save nateraw/9ac46a55825611f4a2c8e8b97df8febb to your computer and use it in GitHub Desktop.
Download ZIP
rit-demo-stable_diffusion_videos.ipynb
Display the source blob
Display the rendered blob
Raw
rit-demo-stable_diffusion_videos.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/nateraw/9ac46a55825611f4a2c8e8b97df8febb/rit-demo-stable_diffusion_videos.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z4GhhH25OdYq"
},
"source": [
"# Stable Diffusion Videos\n",
"\n",
"This notebook allows you to generate videos by interpolating the latent space of [Stable Diffusion](https://github.com/CompVis/stable-diffusion).\n",
"\n",
"You can either dream up different versions of the same prompt, or morph between different text prompts (with seeds set for each for reproducibility).\n",
"\n",
"If you like this notebook:\n",
"- consider giving the [repo a star](https://github.com/nateraw/stable-diffusion-videos) ⭐️\n",
"- consider following me on Github [@nateraw](https://github.com/nateraw) \n",
"\n",
"You can file any issues/feature requests [here](https://github.com/nateraw/stable-diffusion-videos/issues)\n",
"\n",
"Enjoy 🤗"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dvdCBpWWOhW-"
},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Xwfc0ej1L9A0"
},
"outputs": [],
"source": [
"%%capture\n",
"! pip install stable_diffusion_videos accelerate"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "H7UOKJhVOonb"
},
"source": [
"## Run the App 🚀"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "g71hslP8OntM"
},
"source": [
"### Load the Interface\n",
"\n",
"This step will take a couple minutes the first time you run it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "bgSNS368L-DV",
"outputId": "9fa4b15e-0256-4a91-def3-3a020fc7aca5",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 675,
"referenced_widgets": [
"94faf23b0f424b47901a7aca9c7daff9",
"5f04617fe5044b769df88af42b78f7d0",
"465bd3a3deed4641af3409bbd21cd7d2",
"765c10cc0cf448839e099c23ba87627a",
"ff1a02c935f14dbd8cc3b3f1e05d6945",
"e4b2e49061664cbe9522c3e2dedb060c",
"c6399d568d704e37879e95bc3c10a137",
"e19441484f4941a2902735902d85e195",
"a450361c1c9546f1acf12d0ca5bd79b9",
"c00ef14c62664405930d347418d1482f",
"cadede9a1f0347a79652b6f6ad69c940",
"2fe47c05db314da0aaae3ec3e9916180",
"94b8d72060884638bb6b83e65c031ba5",
"4b56154a508b44809224d5c6b6377ce9",
"0c5541ec7b644e4db576342143eddc3c",
"f6981ed038af466e9b1baed4c1cadca5",
"66f7412b3cf04c43a69f0717d273b197",
"8ef5be6ad31d46cb9da06cac9ac2cd36",
"d1a704a9a4ec45ef832f8647641485ac",
"075ab22099f3422e9e88687399960842",
"0813eb86f3ce45da943ff7303216fbc7",
"eec8c9d0d0f748cebbc87bd773a5ceb6",
"3449b14471044077988fd7508a07a8a4",
"67856b6d194844a0bdd73e83f681f1ba",
"426a2e92d0d541c49c0d4db9669e1beb",
"784b5e459761463baece08892f140d3f",
"5f51d0984f70497a8cde351592680b6d",
"56722489f8a047eea4ef5735eb34e105",
"f59bca7603e54b44920c91cbbd3611dd",
"2a3d0d012ec74659ba812875644f71d0",
"4d5c943ab6ed4492b0a3501a632f44c0",
"88f7a1547e8e401f9322d126ec4106e7",
"bd3ec5253f17411394bd970bc487cd17",
"69d67b0f48324239a1239c7d3860af02",
"78118224ebd843b89875761a1d8ae110",
"6d31bacf2fb5425b945e33efdd716632",
"c0f4ef3331ff427baa191c3f1975a2cb",
"33578f7cacd9418a8a14dd69bf4b783f",
"d551b201251244c188e47da1ebb17179",
"c84dcdc7469947e480b8da4a11d0723b",
"1e2d9458f6a04933a04b9703bb149ad8",
"af54993b3a79437aa17e036f2c914a2d",
"f31d9822ec2e478f8b2f3ee2bdf0ac14",
"8ef87e9650f64df4bfc1f742e5a4a0e6",
"b3ea96f9f48d45b08bf643be9902f5b6",
"3a6f38f9267745d3a7b10c280ac642ab",
"43d9f00cb0564963a3136c2201c3b460",
"dc16c3ee03c54da092457716cac8c684",
"b00443d25465438f837abc78e9fcade2",
"03070297580c444a908f160c1044dc3c",
"ea747360c4f14b13955c3f3afd4a8349",
"86d43c94190941fb89c873b8f0210fbe",
"d7ba6c5a4044436c82cc793edc1d67aa",
"5b35382eeb81458a8892c35c7861774d",
"f4553108d7ec43e787e554ef306e9ea6",
"23a9a2e320b34b0eb3596a9bfeb97893",
"84b1578f9d244be2b571e26351b7591b",
"9cac145bc3c44e51a2dd880f390e4681",
"18115d501f9c4345b0101959d8a26e59",
"7e6ee1c048d344c3b0bbb3f339689c4c",
"b1dbf257cac949e8b9070fb088c13c6c",
"54fb2ed190a54751bdf8940581d39286",
"1e0a3e9ea30e470ebb5d97b96e7e4599",
"5583550d2b384ede95367ef45e20a06c",
"0145931f2b3442c8bf8874ddb550d329",
"e4d5e7fa2cd34eb38f9ef3b9ccb92e06",
"4b5c1da0f36942bc8b0228a5e46eb981",
"ba2d2c4c4fe9496184419cc57132868d",
"64fcbe8dd52447a093cc847c17dacbf2",
"656cab7979c74fa69ab2973f9fa694e3",
"53d32e8cfac64caba3d1d043ff405158",
"6e45bda1479844579772815e7a12b3fa",
"68373f98de7e492ab0344887a2db3943",
"4ae1c715a8ae46b085398ad3de0f3ee7",
"a76e0213cde84b63bb82fc5cfa3fffbf",
"4aab3c88b7f1455999de2be07b15d336",
"aa031f5d63d44e0eb2caaf6445400a4d",
"b420b0f797f644f8af78f2d5a7d98b67",
"22512ef78a7349c8bab6364ae6b728f2",
"1739cd888d094613abe023684f25c784",
"49f1aa5984f34676882614b2cc2dea31",
"48d6d2afb11c4b658135dc3ae7b21d11",
"62bd41a4790a49aca909d4328830cf19",
"e658fb6099bc4ef49bad66168043a7f1",
"1cb2cdc58d3543e0a99fbd71304fb256",
"da06d65baeb54030acab4c3dc231663b",
"bcb91ec6899b459c8a9967c9e0ee8feb",
"4c39fb9f0122442fb39023e137afb398",
"28da83ddb91f4819baf55dc56d19145b",
"f7c80cced40b4e93ba0234077141dd8b",
"8daec0f8109f409699dba9f8da60ff55",
"0347359411d94759adc5c909ee4d7127",
"81859b414096452abb618337b2dbe6a3",
"dfcabc3c94674bea88956a809949c292",
"daada17cda0c4aa790e3bbb3ed6e7dff",
"b1199b71cca34d99bb3bfb24a1b58afb",
"171a38cfd67045bb905107d91b3835f6",
"5133cffccc7f4340bf9bdac2d6b78218",
"a4b21266d7454de6a7379cfaf18cd44a",
"b43ba734e9e54f4bbc35077140a8f0a2",
"bf3fe0a27b924b73af630999a092ce76",
"9d1b55698bbb452ca3cfaa3ca861444a",
"1d5b2e60c17042389e0b9bee290a0198",
"dbb3b3d5ce5f420fa033fe7b7b4033ad",
"0269d6985c224ba2a62440f86437548e",
"c1820a03574247d5bc81731cb7eb9864",
"df129aecc0be4f6ea5f2b0dc71d4afdc",
"b9b388468c204bab86ffe9e699f6f235",
"36bff8d04d584b10ac326d35228312a5",
"107d39339b4b4d93b0744ae58edc7904",
"0f44c9236dda42bea4e72ee7c235be84",
"cef73a1094c14e7db0b2f5d4c2b0c282",
"72fddae03d464460a404c6d190dd0185",
"7bf2b703bc2a46cb9d0cbd78e7f0d796",
"bfaadf07cde047329ff0d8a555b4fc68",
"ff97c353b6754cf1af4c8767dbdb5bd4",
"d061d1fa55934b559a2459cfb15befd5",
"addfd2b17c83447fbcfc3b30f0139403",
"441ecd0786fa438a943c571c3ce5b7fd",
"003df7c0cde64d3ea28132d3c566f5ba",
"8e5f4745a2484e1d8aff9c69accaa2de",
"1ac5c7b0389f4552a09bd509d8469811",
"61d63fb6416044ed947953a62b5bb0b3",
"faa87093330440c0a2b17b66158f95eb",
"f6829e2aa9c84a75a99020dcb35f8111",
"d9cd833922d04ab2ac67253cf480dd40",
"d60ba8e734f340b698c81a2f9e9d164e",
"57b05e52e5b941ec8df3b82b8804d344",
"055cc0bfc2674f2f9465100c3df8cdb7",
"ab416459605247558d3689eb355be291",
"736b7655d1374732916e4b7c78ba2454",
"4e5974368f264a0382e528b62aa6e3e4",
"ffd2ae6d491f4d9ba34e5faedfb4931a",
"ed5e111fbdc24d15889bfb3e191162aa",
"807df91edacc41e783fcfeee07246322",
"9d856149ae5d4226aa9e226d9393f772",
"a9c2fa564c8148c282cc2481e09db9e2",
"1a4213b909f74c96b67823314a0580b9",
"5767ebff099d41e2b048c984802491f9",
"96f2ae48830240a8adaf53995a7fcff9",
"1b75811474234c70b39ceb8af702658b",
"b394bf1ad8534c8895537e2ccd5b63c2",
"5afdd448326d4728ba7d7b95bab0db62",
"cf70351b083648c7ac870851ed0f5630",
"b0cefbf1c664457c92b1838e509dcecc",
"935f0b5470f34c1ca2a07692d3b874a7",
"6110e1d2cd504d6fa1b54762ed4b69a5",
"2d2e88f49c864880b501ea07ae71d531",
"8f3104d4e56744ebb66d8522fa1644b4",
"b0d975f05faf451c9e6e7440b81ff000",
"67aa240070a545979c457736c9fe33f7",
"bf0734f66b6f4ffa98c6532195080b41",
"db702716b5e34adfa62e6e64e6cc1a98",
"59041a9279214545b75225d72992885b",
"0d50c60e64de46ed82b9304100c2bdd5",
"546647bff3714ed299c89935a0f00c31",
"8fec70014dac405fa95db16fc5bf3a61",
"2c7029d473944f16b2746aa97b83a6a1",
"a6a1a2ac88724fb7b252471a3c8868d9",
"0949655f3247445e9f78dfc42f453a67",
"94a8cba459b0424c82a4e15453c0c605",
"3fa33c250d174f558c2cc52b16357622",
"41122f6abd364c5285ac1bf18d8c61a6",
"251e36ada2064d56b8a3efab881d4ed2",
"80d687ef10b645d58c13757f0de06339",
"fffd3d27534d4052a429516b9873146d",
"742a217d0b764e2bb88bf7b8ef064f5d",
"16fcaa2aa46549a9bef3d2e9b31c4cc4",
"0fde048ea9214a6cbc49b413c1d4b72e",
"796597eccb584484821b16c473d967b5",
"2cabcf73754d44b893593a4247034737",
"df4fb93a4e8b437b920f5c8e6734fe44",
"ff585efbd4264e26acb931bc7feca130",
"57b2bce152d347099a301a0256a1f95c",
"ad7987e96de44867beba8e80812ad4fd",
"5a77efc83065461594e26c450a5c720b",
"6e43268f0e944e90b8d56be8253a28c7",
"272506e6b38f4da7b845c154bb8988b8",
"a813635a4a2e4f168b0f6c89bceac97f",
"ed5977d282a14b85a87a53c1cf7c1fa2",
"e0c5f977702d42778cf66131c0df9074",
"6af6f87e543246f5a952a69b0983c462",
"806da27392274ec4a7eec7d68c242e43",
"ed124f9d5d3d4aef9e9333b83310292d",
"0f87cf94a2db4e9bbb1b3837b17709b8",
"76c326c044c4433187eedccd117c9e16",
"4ee07f64c3f041c886ff4cc65712f1c0",
"29a2b54437104c37ae17fab06ccc0ceb",
"491bc6b376a04863993920485bb04afe",
"dd2da644584d48a7ada7cc225193a37f",
"ce486c71c9694d61ad8be46783077808",
"932642be61ac459b8afc60d4f5db305d",
"209bfa65d87f49788729dede5a61ee36",
"65d65b165b8b4424a40800ef491edd44",
"1086b5b6ae7a404dbf32b330816d981d",
"c565fd39a1e64572a5b51fc25df2862e",
"c2096ea6de224ef2bf0477a5ebc4059f",
"e1c0a710bc8741dca3757579e685089b",
"deded489c3d44192868acc66e0b81a35",
"ec5d5aef8ae74659a00fc5682c88f251",
"f21e99fc19cb427c9e9d7fe2c221d0b9",
"832a3f54acd64232bfed21c7cc03808a",
"3172b03fa1ea4f74aa9e256ec05eaa48",
"d5062662a8fd4124b3c006d27c34c1b1",
"d8b66cc5b4c04c77a457e9516f624782",
"ceddf49e42084d2983d80fe09b1302a9",
"f5fd32a5c4a348acb6243082706beb64",
"ef44bd9f037440cebb311f70d961392f",
"d46228666c48415895b66f40d5f62e88",
"1e481622341f4d54a82dd85f3080bdb2",
"4d5c689ac8244bf687c65df4adf82d1b",
"149e5141d36f4d00a5c29c638e3421d9",
"4486d83dfab1446bb33541a1e466b45e",
"f5940e376de94a6094dc16d7c0741d48",
"f481bdacead64e1dbac88c7987baa8a5",
"65789eb8d66846d4b59ec1a5410faa2d",
"42a609ac25414b3abda5287e80ddb1e0",
"9106e00d345f4f7b838abf6d22fa16ed",
"080bf4db7edd43e0addd29ba4e9d2dd1",
"3ca7867bb3d94cc093e5a174e40e1672"
]
}
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)ain/model_index.json: 0%| | 0.00/543 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "94faf23b0f424b47901a7aca9c7daff9"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Fetching 19 files: 0%| | 0/19 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "2fe47c05db314da0aaae3ec3e9916180"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading pytorch_model.bin: 0%| | 0.00/492M [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "3449b14471044077988fd7508a07a8a4"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading model.safetensors: 0%| | 0.00/492M [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "69d67b0f48324239a1239c7d3860af02"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading pytorch_model.bin: 0%| | 0.00/1.22G [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "b3ea96f9f48d45b08bf643be9902f5b6"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading model.safetensors: 0%| | 0.00/1.22G [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "23a9a2e320b34b0eb3596a9bfeb97893"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)_checker/config.json: 0%| | 0.00/4.72k [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "4b5c1da0f36942bc8b0228a5e46eb981"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)rocessor_config.json: 0%| | 0.00/342 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "b420b0f797f644f8af78f2d5a7d98b67"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)_encoder/config.json: 0%| | 0.00/617 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "28da83ddb91f4819baf55dc56d19145b"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)cheduler_config.json: 0%| | 0.00/308 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "b43ba734e9e54f4bbc35077140a8f0a2"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)cial_tokens_map.json: 0%| | 0.00/472 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "0f44c9236dda42bea4e72ee7c235be84"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)okenizer_config.json: 0%| | 0.00/806 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "1ac5c7b0389f4552a09bd509d8469811"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)tokenizer/merges.txt: 0%| | 0.00/525k [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "ffd2ae6d491f4d9ba34e5faedfb4931a"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)tokenizer/vocab.json: 0%| | 0.00/1.06M [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "cf70351b083648c7ac870851ed0f5630"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)on_pytorch_model.bin: 0%| | 0.00/3.44G [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "0d50c60e64de46ed82b9304100c2bdd5"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)ch_model.safetensors: 0%| | 0.00/3.44G [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "fffd3d27534d4052a429516b9873146d"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)819/unet/config.json: 0%| | 0.00/743 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "6e43268f0e944e90b8d56be8253a28c7"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)d819/vae/config.json: 0%| | 0.00/547 [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "29a2b54437104c37ae17fab06ccc0ceb"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)on_pytorch_model.bin: 0%| | 0.00/335M [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "deded489c3d44192868acc66e0b81a35"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading (…)ch_model.safetensors: 0%| | 0.00/335M [00:00<?, ?B/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "1e481622341f4d54a82dd85f3080bdb2"
}
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config[\"id2label\"]` will be overriden.\n"
]
}
],
"source": [
"from stable_diffusion_videos import StableDiffusionWalkPipeline, Interface\n",
"import torch\n",
"\n",
"device = \"mps\" if torch.backends.mps.is_available() else \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"torch_dtype = torch.float16 if device == \"cuda\" else torch.float32\n",
"pipeline = StableDiffusionWalkPipeline.from_pretrained(\n",
" \"runwayml/stable-diffusion-v1-5\",\n",
" torch_dtype=torch_dtype,\n",
").to(device)\n",
"\n",
"interface = Interface(pipeline)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "kidtsR3c2P9Z"
},
"outputs": [],
"source": [
"#@title Connect to Google Drive to Save Outputs\n",
"\n",
"#@markdown If you want to connect Google Drive, click the checkbox below and run this cell. You'll be prompted to authenticate.\n",
"\n",
"#@markdown If you just want to save your outputs in this Colab session, don't worry about this cell\n",
"\n",
"connect_google_drive = True #@param {type:\"boolean\"}\n",
"\n",
"#@markdown Then, in the interface, use this path as the `output` in the Video tab to save your videos to Google Drive:\n",
"\n",
"#@markdown > /content/gdrive/MyDrive/stable_diffusion_videos\n",
"\n",
"\n",
"if connect_google_drive:\n",
" from google.colab import drive\n",
"\n",
" drive.mount('/content/gdrive')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VxjRVNnMOtgU"
},
"source": [
"### Launch\n",
"\n",
"This cell launches a Gradio Interface. Here's how I suggest you use it:\n",
"\n",
"1. Use the \"Images\" tab to generate images you like.\n",
" - Find two images you want to morph between\n",
" - These images should use the same settings (guidance scale, height, width)\n",
" - Keep track of the seeds/settings you used so you can reproduce them\n",
"\n",
"2. Generate videos using the \"Videos\" tab\n",
" - Using the images you found from the step above, provide the prompts/seeds you recorded\n",
" - Set the `num_interpolation_steps` - for testing you can use a small number like 3 or 5, but to get great results you'll want to use something larger (60-200 steps). \n",
"\n",
"💡 **Pro tip** - Click the link that looks like `https://<id-number>.gradio.app` below , and you'll be able to view it in full screen."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8es3_onUOL3J",
"outputId": "f2ea98fe-e003-4894-e535-0d3335597b02",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 930,
"referenced_widgets": [
"af8be400bf114893865bc768334b3ff7",
"f14f517a063946f58f98fc1a8bf38f66",
"e16327952112480782097ebd33a3b136",
"20249c41b968417a80b900be89a840d3",
"53e5b92e308e4accb4c94e7be37be371",
"04888752d65b4d8c930d03d254fdc53a",
"0866f0e65268416fb9d1ee0d8ff622e6",
"5065a831577641f380ae56a0cdc7062f",
"900ead8d6f0a4c2bacc16f79d071235d",
"1666a20d9124421eb4d2decbdb06889b",
"dc9c70dc13664ab397ce2cdb3b0329ab",
"446769740ffa4aa0a46365c7e6346a82",
"bf90bc3f99b84c27a8c607c364a19539",
"b5f32dae245c464f8f883a72eab93bf5",
"208d745b8d7145dfb53fdd9ae249faa7",
"6b61dde25663437b889305c6b917af6f",
"d502cdeb06a94544bd6a2b7059b441e9",
"5a8167113934438f9b5de0d63c148819",
"3891dfddab754222855b81bee09366f0",
"8059177fb0894beaaaf94f6299ed44cb",
"89eee8a7ffec4bbd8428de646c8e2224",
"524c3ed881c843248d1de193b03d7bc3",
"e20f081c53954d3eb7797fd14eed7876",
"e3da9df5cfeb4d72862bf1869359a2f6",
"f0cb99074c874aff8aa05a0bdff0c42d",
"d837ed24cce24614a30ad0a9365ee1f8",
"e2f510d9e88b48338bc9db7b071c9a28",
"963100674938464bb29fc9ddebf2eb87",
"925623da16fa4930905c4ebbf93fa3cd",
"4ad8609cb0ef414e8bcce8e6128556ad",
"dfe759ce2c47466ab3c9c9ec66d80495",
"ca6f4dfb15cc4bfa98bdd316cf3dfec5",
"db5714569a5e452da0b0894e12f72b2c",
"b29c8112e6e94ff7acff268a266e2c38",
"bfec6420a0984ef3a467a41073465ed3",
"f6b8c7281b2b46f1b6c098af37b7e531",
"1df89b01092f4012b460e921863fd29b",
"f0319239d5d24dd1a7035c58ef9256ff",
"4ad86bbddfa942ef8654f0bfb7565633",
"5a2bce9147ae439596a3afc87c5f3c77",
"5677a3714c644add9fd63b5c9fc27abf",
"a182e161ea234d6eaf601e2229deb070",
"ad132fc6e61c48138599fc2fd3342d90",
"5758137c3a1a430a82f0b27b5e2458f5",
"b97ec80d0d2b42e698464bde8b1fdb7f",
"630471bac5ce4b0197cae0c47e321b4b",
"94cc15b8c1c24b49a8e39c0c2e9df922",
"b0b92bf71fca488c8c7603ab4b0b4ca5",
"8f246ecac825492f8fb3ccf2a6386423",
"e0161610642c44438b3fb6607bfd290a",
"5f2c5ee8d8974cd486b8e7e635ed6c4e",
"ffa9b8a6a4234003b184aa206d2709aa",
"7771a6bc851d439a85678cb861678786",
"3405d42932414cfaab78515236e802f6",
"0620165f780842c0b35ad7cdf3163298",
"8d67b6100a0a4049bd5a31a81992a378",
"49a459b3222b4c4f84260b5dc9e66bf9",
"af8d8f3ccd4147bf8d8fc782ef134010",
"6deec29a50b746399c6d2d03335024eb",
"a2db09a9cf354b588a29f0547a4f2e71",
"39a33d0de0e348efbff55a1ccd02a002",
"fdbedcfeef024326a6aa2c0a9e924966",
"469e7c8d60644f4bbeaa21b9ae263a28",
"1b824328ee384638bddcc79afa8c1744",
"80f9b560b1ca476abb61ee3dc8b8a5b1",
"8f85446002d445e78bc1398329b510e3",
"6226fea2455e4da3bdf10433beb75938",
"3521d90d94de4776ba4de23a880ef117",
"dd269ab564834b46a886356bf4fca269",
"5932c37e232046cc946b25d848ab150d",
"e07e54317e8d474793eb207ecd4b7a92",
"74cf956167684f8e946af559c7d1137f",
"004472275750460a835e56ea00dac419",
"76f59cdc411640199f51b7b80285511e",
"83084d96226c498396adf41164616917",
"17fef62fd7234fb29dcbc10daec391c6",
"54ae1c1332f442d0a86d077244f5e081"
]
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Colab notebook detected. This cell will run indefinitely so that you can see errors and logs. To turn off, set debug=False in launch().\n",
"Note: opening Chrome Inspector may crash demo inside Colab notebooks.\n",
"\n",
"To create a public link, set `share=True` in `launch()`.\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<IPython.core.display.Javascript object>"
],
"application/javascript": [
"(async (port, path, width, height, cache, element) => {\n",
" if (!google.colab.kernel.accessAllowed && !cache) {\n",
" return;\n",
" }\n",
" element.appendChild(document.createTextNode(''));\n",
" const url = await google.colab.kernel.proxyPort(port, {cache});\n",
"\n",
" const external_link = document.createElement('div');\n",
" external_link.innerHTML = `\n",
" <div style=\"font-family: monospace; margin-bottom: 0.5rem\">\n",
" Running on <a href=${new URL(path, url).toString()} target=\"_blank\">\n",
" https://localhost:${port}${path}\n",
" </a>\n",
" </div>\n",
" `;\n",
" element.appendChild(external_link);\n",
"\n",
" const iframe = document.createElement('iframe');\n",
" iframe.src = new URL(path, url).toString();\n",
" iframe.height = height;\n",
" iframe.allow = \"autoplay; camera; microphone; clipboard-read; clipboard-write;\"\n",
" iframe.width = width;\n",
" iframe.style.border = 0;\n",
" element.appendChild(iframe);\n",
" })(7860, \"/\", \"100%\", 500, false, window.element)"
]
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Generating batch 0\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "af8be400bf114893865bc768334b3ff7"
}
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Generating batch 0\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "446769740ffa4aa0a46365c7e6346a82"
}
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Generating batch 0\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "e20f081c53954d3eb7797fd14eed7876"
}
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Generating batch 0\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "b29c8112e6e94ff7acff268a266e2c38"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "b97ec80d0d2b42e698464bde8b1fdb7f"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "8d67b6100a0a4049bd5a31a81992a378"
}
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/51 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "6226fea2455e4da3bdf10433beb75938"
}
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Keyboard interruption in main thread... closing server.\n"
]
}
],
"source": [
"interface.launch(debug=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mFCoTvlnPi4u"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SjTQLCiLOWeo"
},
"source": [
"## Use `walk` programmatically\n",
"\n",
"The other option is to not use the interface, and instead use `walk` programmatically. Here's how you would do that..."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fGQPClGwOR9R"
},
"source": [
"First we define a helper fn for visualizing videos in colab"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "GqTWc8ZhNeLU"
},
"outputs": [],
"source": [
"from IPython.display import HTML\n",
"from base64 import b64encode\n",
"\n",
"def visualize_video_colab(video_path):\n",
" mp4 = open(video_path,'rb').read()\n",
" data_url = \"data:video/mp4;base64,\" + b64encode(mp4).decode()\n",
" return HTML(\"\"\"\n",
" <video width=400 controls>\n",
" <source src=\"%s\" type=\"video/mp4\">\n",
" </video>\n",
" \"\"\" % data_url)"
]
},
{
"cell_type": "code",
"source": [
"visualize_video_colab(\"/content/dreams/20230406-163738/20230406-163738.mp4\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 421
},
"id": "-PeILafwb33d",
"outputId": "bfe13d42-5c5a-4d2a-8176-5aef68b916f9"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<IPython.core.display.HTML object>"
],
"text/html": [
"\n",
" <video width=400 controls>\n",
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment