README change

.idea/.gitignore

@@ -0,0 +1,8 @@
+# Default ignored files
+/shelf/
+/workspace.xml
+# Datasource local storage ignored files
+/dataSources/
+/dataSources.local.xml
+# Editor-based HTTP Client requests
+/httpRequests/

.idea/inspectionProfiles/Project_Default.xml

@@ -0,0 +1,18 @@
+<component name="InspectionProjectProfileManager">
+  <profile version="1.0">
+    <option name="myName" value="Project Default" />
+    <inspection_tool class="Eslint" enabled="true" level="WARNING" enabled_by_default="true" />
+    <inspection_tool class="PyPackageRequirementsInspection" enabled="true" level="WARNING" enabled_by_default="true">
+      <option name="ignoredPackages">
+        <value>
+          <list size="4">
+            <item index="0" class="java.lang.String" itemvalue="tensorboard" />
+            <item index="1" class="java.lang.String" itemvalue="shapely" />
+            <item index="2" class="java.lang.String" itemvalue="geopandas" />
+            <item index="3" class="java.lang.String" itemvalue="sklearn" />
+          </list>
+        </value>
+      </option>
+    </inspection_tool>
+  </profile>
+</component>

.idea/inspectionProfiles/profiles_settings.xml

@@ -0,0 +1,6 @@
+<component name="InspectionProjectProfileManager">
+  <settings>
+    <option name="USE_PROJECT_PROFILE" value="false" />
+    <version value="1.0" />
+  </settings>
+</component>

.idea/misc.xml

@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project version="4">
+  <component name="ProjectRootManager" version="2" project-jdk-name="Remote Python 3.8.8 (sftp://[email protected]:8022/home/stella/anaconda3/envs/python3.8-pytorch17.1/bin/python)" project-jdk-type="Python SDK" />
+</project>

.idea/modules.xml

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project version="4">
+  <component name="ProjectModuleManager">
+    <modules>
+      <module fileurl="file://$PROJECT_DIR$/.idea/putting-nerf-on-a-diet.iml" filepath="$PROJECT_DIR$/.idea/putting-nerf-on-a-diet.iml" />
+    </modules>
+  </component>
+</project>

.idea/putting-nerf-on-a-diet.iml

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<module type="PYTHON_MODULE" version="4">
+  <component name="NewModuleRootManager">
+    <content url="file://$MODULE_DIR$" />
+    <orderEntry type="inheritedJdk" />
+    <orderEntry type="sourceFolder" forTests="false" />
+  </component>
+</module>

.idea/vcs.xml

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project version="4">
+  <component name="VcsDirectoryMappings">
+    <mapping directory="$PROJECT_DIR$" vcs="Git" />
+  </component>
+</project>

README.md

@@ -1,16 +1,13 @@
 # Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis Implementation
+###** WARNING : it is not the completed REAME (Until Thursday)**
+<p align="center"><img width="450" alt="스크린샷 2021-07-04 오후 4 11 51" src="https://user-images.githubusercontent.com/77657524/126361638-4aad58e8-4efb-4fc5-bf78-f53d03799e1e.png"></p>
 
-<img width="586" alt="스크린샷 2021-07-04 오후 4 11 51" src="https://user-images.githubusercontent.com/77657524/124376591-b312b780-dce2-11eb-80ad-9129d6f5eedb.png">
+the Pytorch, JAX/Flax based code implementation of this paper [Putting NeRF on a Diet : Ajay Jain, Matthew Tancik, Pieter Abbeel, Arxiv : https://arxiv.org/abs/2104.00677] 
+The model generates the novel view synthesis redering (NeRF: Neural Radiances Field) base on Fewshot learning.
+The semantic loss using pre-trained CLIP Vision Transformer embedding is used for 2D supervision for 3D. It outperforms the Original NeRF in 3D reconstruction for 
 
-the Pytorch, JAX/Flax based code implementation of this paper : https://arxiv.org/abs/2104.00677
-The model gives the 3D neural scene representation (NeRF: Neural Radiances Field) estimated from a few images. 
-Which is based on extracting the semantic information using a pre-trained visual encoder such as CLIP, a Vision Transformer
-
-Our Project is started in the HuggingFace X GoogleAI (JAX) Community Week Event.
-https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104
-
-##  Hugging Face Hub Repo URL: 
 
+##  🤗 Hugging Face Hub Repo URL:
 We will also upload our project on the Hugging Face Hub Repository. 
 [https://huggingface.co/flax-community/putting-nerf-on-a-diet/](https://huggingface.co/flax-community/putting-nerf-on-a-diet/)
 
@@ -49,7 +46,7 @@ Our JAX/Flax implementation currently supports:
 </tbody>
 </table>
 
-## Installation
+## 💻 Installation
 
 ```bash
 # Clone the repo
@@ -68,12 +65,27 @@ pip install --upgrade jax jaxlib==0.1.57+cuda101 -f https://storage.googleapis.c
 pip install flax transformer[flax]
 ```
 
-## Dataset
+## ⚽ Dataset & Methods
 Download the datasets from the [NeRF official Google Drive](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1).
 Please download the `nerf_synthetic.zip` and unzip them
 in the place you like. Let's assume they are placed under `/tmp/jaxnerf/data/`.
 
-## How to use
+<p align="center"><img width="400" alt="스크린샷 2021-07-04 오후 4 11 51" src="https://user-images.githubusercontent.com/77657524/124376591-b312b780-dce2-11eb-80ad-9129d6f5eedb.png"></p> 
+
+Based on the principle
+that “a bulldozer is a bulldozer from any perspective”, our proposed DietNeRF supervises the radiance field from arbitrary poses
+(DietNeRF cameras). This is possible because we compute a semantic consistency loss in a feature space capturing high-level
+scene attributes, not in pixel space. We extract semantic representations of renderings using the CLIP Vision Transformer, then
+maximize similarity with representations of ground-truth views. In
+effect, we use prior knowledge about scene semantics learned by
+single-view 2D image encoders to constrain a 3D representation.
+
+You can check detail information on the author's paper. Also, you can check the CLIP based semantic loss structure on the following image.
+<p align="center"><img width="600" alt="스크린샷 2021-07-04 오후 4 11 51" src="https://user-images.githubusercontent.com/77657524/126386709-a4ce7ff8-2a68-442f-b4ed-26971fb90e51.png"></p>
+
+Our code used JAX/FLAX framework for implementation. So that it can achieve much speed up than other NeRF code. Moreover, we implemented multiple GPU distribution ray code. it helps much smaller training time. At last, our code used hugging face, transformer,  CLIP model library. 
+
+## 🤟 How to use
 ```
 python -m train \
   --data_dir=/PATH/TO/YOUR/SCENE/DATA \ % e.g., nerf_synthetic/lego
@@ -82,33 +94,78 @@ python -m train \
 ```
 You can toggle the semantic loss by “use_semantic_loss” in configuration files.
 
-## Rendered examples by 8-shot learned Diet-NeRF
-- Lego
-- Chair
+## 💎 Performance
+
+### Performance Tables
+#### 4 Shot Blender Dataset PSNR Result
+
+| Scene   |   Chair   |   Drums   |   Ficus   |   Hotdog  |    Lego   | Materials |    Mic    |    Ship   |    Mean   |
+|---------|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|
+| NeRF |   33.00   |   25.01   |   30.13   |   36.18   |   32.54   |   29.62   |   32.91   |   28.65   |   31.01   |
+| DietNeRF | **34.08** | **25.03** | **30.43** | **36.92** | **33.28** | **29.91** | **34.53** | **29.36** | **31.69** |
+
+#### Loss Graph Comparison btw NeRF vs DietNeRF in Drum Scene
+
+<p align="center"><img width="400" alt="스크린샷 2021-07-04 오후 4 11 51" src="https://user-images.githubusercontent.com/77657524/126384510-423b9070-a3e5-4e18-8b4e-30c15c5b39c6.png">
+</p>
+
+
+### - Rendering GIF images by 8-shot learned Diet-NeRF
+
+DietNeRF has a strong capacity to generalise on novel and challenging views with EXTREMELY SMALL TRAINING SAMPLES!  
+The animations below shows the performance difference between DietNeRF (left) v.s. NeRF (right) with only 4 training images: 
+
+#### SHIP
+![Text](./assets/ship-dietnerf.gif) ![Alt Text](./assets/ship-nerf.gif)
+
+#### LEGO
+![Text](./assets/ship-dietnerf.gif) ![Alt Text](./assets/ship-nerf.gif)
+
+#### HOTDOG
+![Text](./assets/ship-dietnerf.gif) ![Alt Text](./assets/ship-nerf.gif)
 
-## Rendered examples by occluded 14-shot learned NeRF and Diet-NeRF
+
+### - Rendered Rendering images by 4-shot learned Diet-NeRF vs Vanilla-NeRF
+
+#### SHIP
+@ will be filled
+
+#### LEGO
+@ will be filled
+
+#### HOTDOG
+@ will be filled
+
+### - Rendered examples by occluded 14-shot learned NeRF and Diet-NeRF
 This result is on the quite initial state and expected to be improved.
 
-### Training poses
+#### Training poses
 <img width="1400" src="https://user-images.githubusercontent.com/26036843/126111980-4f332c87-a7f0-42e0-a355-8e77621bbca4.png">
 
-### Rendered novel poses
+#### Rendered novel poses
 <img width="800" src="https://user-images.githubusercontent.com/26036843/126113080-a6a48f3d-2629-4efc-a740-fe908ca6b5c3.png">
 
 
-## Demo
+## 🤩 Demo
+
+You can check our Streamlit Space Demo on following site !
 [https://huggingface.co/spaces/flax-community/DietNerf-Demo](https://huggingface.co/spaces/flax-community/DietNerf-Demo)
 
-## Our Teams
+## 👨‍👧‍👦 Our Teams
+
 
 | Teams            | Members                                                                                                                                                        |
 |------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| NeRF Team        | Leader : [JaeYoung Chung](https://github.com/robot0321), Members : [Stella Yang](https://github.com/codestella), [Alex Lau](https://github.com/riven314), [Haswanth Aekula](https://github.com/hassiahk), [Hyunkyu Kim](https://github.com/minus31)           |
-| CLIP Team        | Leader : [Sunghyun Kim](https://github.com/MrBananaHuman), Members : [Seunghyun Lee](https://github.com/sseung0703), [Sasikanth Kotti](https://github.com/ksasi), [Khali Sifullah](https://github.com/khalidsaifullaah)                                 |
-| Cloud TPU Team   | Leader : [Alex Lau](https://github.com/riven314), Members : [JaeYoung Chung](https://github.com/robot0321), [Sunghyun Kim](https://github.com/MrBananaHuman), [Aswin Pyakurel](https://github.com/masapasa)                                                     |
 | Project Managing | [Stella Yang](https://github.com/codestella) To Watch Our Project Progress, Please Check [Our Project Notion](https://www.notion.so/Putting-NeRF-on-a-Diet-e0caecea0c2b40c3996c83205baf870d) |
+| NeRF Team        | [Stella Yang](https://github.com/codestella), [Alex Lau](https://github.com/riven314), [Seunghyun Lee](https://github.com/sseung0703), [Hyunkyu Kim](https://github.com/minus31),  [Haswanth Aekula](https://github.com/hassiahk), [JaeYoung Chung](https://github.com/robot0321)          |
+| CLIP Team        | [Seunghyun Lee](https://github.com/sseung0703), [Sasikanth Kotti](https://github.com/ksasi), [Khali Sifullah](https://github.com/khalidsaifullaah) , [Sunghyun Kim](https://github.com/MrBananaHuman)                                |
+| Cloud TPU Team   | [Alex Lau](https://github.com/riven314), [Aswin Pyakurel](https://github.com/masapasa) , [JaeYoung Chung](https://github.com/robot0321),  [Sunghyun Kim](https://github.com/MrBananaHuman)                                                    |
+
+* Extremely Don't Sleep Contributors 🤣 :  [Seunghyun Lee](https://github.com/sseung0703), [Alex Lau](https://github.com/riven314), [Stella Yang](https://github.com/codestella)
+
 
-## References
+
+## 🌱 References
 This project is based on “JAX-NeRF”.
 ```
 @software{jaxnerf2020github,
@@ -132,5 +189,17 @@ This project is based on “JAX-NeRF”.
 }
 ```
 
-## License
-[Apache License 2.0](https://github.com/codestella/putting-nerf-on-a-diet/blob/main/LICENSE)
+## 🔑 License
+[Apache License 2.0](https://github.com/codestella/putting-nerf-on-a-diet/blob/main/LICENSE)
+
+## ❤️ Special Thanks 
+
+
+Our Project is started in the HuggingFace X GoogleAI (JAX) Community Week Event.
+https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104
+
+Thank you for Our Mentor Suraj and Organizers in JAX/Flax Community Week! 
+Our team grows up with this community learning experience. It was wonderful time!
+
+<p align="center"><img width="250" alt="스크린샷 2021-07-04 오후 4 11 51" src="https://user-images.githubusercontent.com/77657524/126369170-5664076c-ac99-4157-bc53-b91dfb7ed7e1.jpeg"></p>
+