Upload 7 files

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+vid-1.mp4 filter=lfs diff=lfs merge=lfs -text
+vid-2.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,14 +1,231 @@
----
-title: Video Anamoly Detector
-emoji: 👀
-colorFrom: gray
-colorTo: yellow
-sdk: streamlit
-sdk_version: 1.43.1
-app_file: app.py
-pinned: false
-license: apache-2.0
-short_description: 'Scan cctv footage to detect suspicious activity '
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+---
+title: Video Anomaly Detector
+emoji: 🎥
+colorFrom: blue
+colorTo: green
+sdk: streamlit
+sdk_version: 1.31.0
+app_file: app.py
+pinned: false
+license: mit
+---
+
+# Video Anomaly Detector
+
+This application analyzes video files frame by frame using advanced AI models to detect anomalies based on a user-provided prompt.
+
+## Model Description
+
+The application supports multiple AI models for analysis:
+
+- **GPT-4o**: OpenAI's most powerful multimodal model, offering the highest accuracy for anomaly detection
+- **GPT-4o-mini**: A smaller, faster, and more cost-effective version of GPT-4o
+- **Phi-4**: Microsoft's multimodal model that can run locally using Hugging Face transformers
+- **Phi-3**: *(Coming soon)* Microsoft's earlier multimodal model
+
+Each model can analyze both text and images, examining video frames to identify potential anomalies based on the user's prompt.
+
+## Demo App
+
+[![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://huggingface.co/spaces/username/video-anomaly-detector)
+
+## Features
+
+- Support for both video files and live streams (webcam, IP camera, RTSP)
+- Select from multiple AI models (GPT-4o, GPT-4o-mini, Phi-4)
+- Skip frames for faster processing
+- Provide custom prompts for anomaly detection
+- Two analysis modes: frame-by-frame or cumulative summary
+- Batch processing for multiple videos
+- Streamlit web interface with modern UI design
+
+## How It Works
+
+1. The application extracts frames from the uploaded video or live stream
+2. It skips a user-defined number of frames to reduce processing time
+3. Based on the selected analysis depth:
+   - **Granular mode**: Each selected frame is analyzed individually
+   - **Cumulative mode**: All frames are analyzed together to provide an overall summary
+4. The selected AI model analyzes the frame(s) and provides descriptions of any detected anomalies
+5. Results are displayed in an interactive interface with timestamps for live streams
+
+## Requirements
+
+- Python 3.8+
+- OpenAI API key with access to GPT-4o and GPT-4o-mini models (only needed for OpenAI models)
+- For Phi-4: GPU recommended but not required (will use CPU if GPU not available)
+- For live streams: Webcam or access to an IP camera/RTSP stream
+
+## Installation
+
+```bash
+git clone https://github.com/username/video-anomaly-detector.git
+cd video-anomaly-detector
+pip install -r requirements.txt
+```
+
+## Environment Variables
+
+Create a `.env` file in the root directory with your OpenAI API key (only needed for OpenAI models):
+
+```
+OPENAI_API_KEY=your_openai_api_key_here
+```
+
+## Usage
+
+### Web Application
+
+Run the Streamlit application:
+
+```bash
+streamlit run app.py
+```
+
+Your browser will automatically open with the application running at http://localhost:8501
+
+#### Using with Video Files
+
+1. Select "Video File" as the input source
+2. Upload a video file
+3. Configure the analysis settings
+4. Click "Analyze Video"
+
+#### Using with Live Streams
+
+1. Select "Live Stream" as the input source
+2. Choose between "Webcam" or "IP Camera / RTSP Stream"
+3. For IP cameras, enter the stream URL (e.g., rtsp://username:password@ip_address:port/path)
+4. Set the maximum number of frames to process
+5. Configure the analysis settings
+6. Click "Analyze Video"
+
+### Command Line
+
+#### Single Video Processing
+
+```bash
+python example.py --video path/to/video.mp4 --skip 5 --analysis_depth granular --model gpt-4o --prompt "Detect any unusual activities or objects in this frame"
+```
+
+Arguments:
+- `--video`: Path to the video file (required)
+- `--skip`: Number of frames to skip (default: 5)
+- `--analysis_depth`: Analysis depth: 'granular' or 'cumulative' (default: 'granular')
+- `--model`: AI model to use: 'gpt-4o', 'gpt-4o-mini', or 'phi-4' (default: 'gpt-4o')
+- `--prompt`: Prompt for anomaly detection
+- `--api_key`: OpenAI API key (optional if set in .env file, not needed for Phi-4)
+
+#### Live Stream Processing
+
+```bash
+python example.py --stream 0 --skip 5 --analysis_depth granular --model gpt-4o --max_frames 30 --prompt "Detect any unusual activities or objects in this frame"
+```
+
+Arguments:
+- `--stream`: Stream source (0 for webcam, URL for IP camera/RTSP stream)
+- `--max_frames`: Maximum number of frames to process (default: 30)
+- Other arguments are the same as for video processing
+
+#### Batch Processing
+
+```bash
+python batch_process.py --videos_dir path/to/videos --output_dir output --skip 5 --analysis_depth cumulative --model gpt-4o-mini
+```
+
+Arguments:
+- `--videos_dir`: Directory containing video files (required)
+- `--output_dir`: Directory to save results (default: 'output')
+- `--skip`: Number of frames to skip (default: 5)
+- `--analysis_depth`: Analysis depth: 'granular' or 'cumulative' (default: 'granular')
+- `--model`: AI model to use: 'gpt-4o', 'gpt-4o-mini', or 'phi-4' (default: 'gpt-4o')
+- `--prompt`: Prompt for anomaly detection
+- `--api_key`: OpenAI API key (optional if set in .env file, not needed for Phi-4)
+- `--extensions`: Comma-separated list of video file extensions (default: '.mp4,.avi,.mov,.mkv')
+
+## Model Options
+
+### GPT-4o
+- OpenAI's most powerful multimodal model
+- Highest accuracy for anomaly detection
+- Requires OpenAI API key
+- Recommended for critical applications where accuracy is paramount
+
+### GPT-4o-mini
+- Smaller, faster version of GPT-4o
+- More cost-effective for processing large videos
+- Requires OpenAI API key
+- Good balance between performance and cost
+
+### Phi-4
+- Microsoft's multimodal model
+- Runs locally using Hugging Face transformers
+- No API key required
+- First run will download the model (approximately 5GB)
+- GPU recommended but not required
+
+### Phi-3 (Coming Soon)
+- Microsoft's earlier multimodal model
+- Will provide an alternative option for analysis
+
+## Analysis Depth Options
+
+### Granular - Frame by Frame
+- Analyzes each frame individually
+- Provides detailed analysis for every processed frame
+- Useful for detecting specific moments or events
+
+### Cumulative - All Frames
+- Analyzes all frames together to provide an overall summary
+- Identifies up to 3 key frames that best represent detected anomalies
+- Useful for getting a high-level understanding of anomalies in the video
+
+## Deploying to Hugging Face Spaces
+
+This project is configured for easy deployment to Hugging Face Spaces:
+
+1. Fork this repository to your GitHub account
+2. Create a new Space on Hugging Face: https://huggingface.co/spaces/create
+3. Select "Streamlit" as the SDK
+4. Link your GitHub repository
+5. Add your OpenAI API key as a secret in the Space settings (if using OpenAI models)
+6. The Space will automatically deploy with the configuration from this repository
+
+Alternatively, you can use the GitHub Actions workflow to automatically sync your repository to Hugging Face Spaces:
+
+1. Create a Hugging Face account and generate an access token
+2. Add the following secrets to your GitHub repository:
+   - `HF_TOKEN`: Your Hugging Face access token
+   - `HF_USERNAME`: Your Hugging Face username
+   - `OPENAI_API_KEY`: Your OpenAI API key (if using OpenAI models)
+3. Push to the main branch to trigger the workflow
+
+## Project Structure
+
+```
+video-anomaly-detector/
+├── app.py                # Streamlit web application
+├── detector.py           # Core video processing and anomaly detection with OpenAI models
+├── phi4_detector.py      # Phi-4 model implementation using Hugging Face
+├── example.py            # Example script for processing a single video
+├── batch_process.py      # Script for batch processing multiple videos
+├── requirements.txt      # Python dependencies
+├── requirements-hf.txt   # Dependencies for Hugging Face Spaces
+├── .env.example          # Template for environment variables
+└── .github/              # GitHub Actions workflows
+    └── workflows/
+        └── sync-to-hub.yml  # Workflow to sync to Hugging Face
+```
+
+## Limitations
+
+- Processing time depends on the video length, frame skip rate, and your internet connection
+- The OpenAI models require an API key and may incur usage costs
+- Phi-4 model requires downloading approximately 5GB of model files on first use
+- Higher frame skip values will process fewer frames, making analysis faster but potentially less accurate
+- Cumulative analysis may miss some details that would be caught in granular analysis
+- Live stream processing may be affected by network latency and camera quality
+
+## License
+
+This project is licensed under the MIT License - see the LICENSE file for details. 

app.py

@@ -0,0 +1,1240 @@
+"""
+---
+title: Video Anomaly Detector
+emoji: 🎥
+colorFrom: blue
+colorTo: green
+sdk: streamlit
+sdk_version: 1.31.0
+app_file: app.py
+pinned: false
+license: mit
+---
+"""
+import streamlit as st
+import os
+import tempfile
+import time
+from detector import VideoAnomalyDetector
+import cv2
+from PIL import Image
+import numpy as np
+from dotenv import load_dotenv
+import streamlit.components.v1 as components
+import json
+import base64
+from io import BytesIO
+import smtplib
+from email.mime.text import MIMEText
+from email.mime.multipart import MIMEMultipart
+from email.mime.image import MIMEImage
+import requests
+import re
+
+# Custom JSON encoder to handle numpy arrays and other non-serializable types
+class NumpyEncoder(json.JSONEncoder):
+    def default(self, obj):
+        if isinstance(obj, np.ndarray):
+            # Convert numpy arrays to base64 encoded strings
+            pil_img = Image.fromarray(obj)
+            buffered = BytesIO()
+            pil_img.save(buffered, format="PNG")
+            img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
+            return {"__ndarray__": img_str}
+        return super(NumpyEncoder, self).default(obj)
+
+
+
+
+def send_email_notification(to_email, subject, body, image=None):
+    """Send email notification with optional image attachment"""
+    try:
+        # Get email credentials from environment variables
+        smtp_server = os.getenv("SMTP_SERVER", "smtp.gmail.com")
+        smtp_port = int(os.getenv("SMTP_PORT", "587"))
+        smtp_username = os.getenv("SMTP_USERNAME")
+        smtp_password = os.getenv("SMTP_PASSWORD")        
+        
+        if not smtp_username or not smtp_password:
+            st.warning("Email notification failed: SMTP credentials not configured. Please set SMTP_USERNAME and SMTP_PASSWORD environment variables.")
+            return False
+        
+        # Create message
+        msg = MIMEMultipart()
+        msg['From'] = smtp_username
+        msg['To'] = to_email
+        msg['Subject'] = subject
+        
+        # Attach text
+        msg.attach(MIMEText(body, 'plain'))
+        
+        # Attach image if provided
+        if image is not None:
+            # Convert numpy array to image
+            if isinstance(image, np.ndarray):
+                pil_img = Image.fromarray(image)
+                img_byte_arr = BytesIO()
+                pil_img.save(img_byte_arr, format='PNG')
+                img_data = img_byte_arr.getvalue()
+            else:
+                # Assume it's already bytes
+                img_data = image
+                
+            img_attachment = MIMEImage(img_data)
+            img_attachment.add_header('Content-Disposition', 'attachment', filename='anomaly.png')
+            msg.attach(img_attachment)
+        
+        # Connect to server and send
+        server = smtplib.SMTP(smtp_server, smtp_port)
+        server.starttls()
+        server.login(smtp_username, smtp_password)
+        server.send_message(msg)
+        server.quit()
+        
+        return True
+    except Exception as e:
+        st.warning(f"Email notification failed: {str(e)}")
+        return False
+
+def send_whatsapp_notification(to_number, message):
+    """Send WhatsApp notification using WhatsApp Business API"""
+    try:
+        # Get WhatsApp API credentials from environment variables
+        whatsapp_api_key = os.getenv("WHATSAPP_API_KEY")
+        whatsapp_phone_id = os.getenv("WHATSAPP_PHONE_ID")
+        
+        if not whatsapp_api_key or not whatsapp_phone_id:
+            st.warning("WhatsApp notification failed: API credentials not configured. Please set WHATSAPP_API_KEY and WHATSAPP_PHONE_ID environment variables.")
+            return False
+        
+        # For demonstration purposes, we'll show how to use the WhatsApp Business API
+        # In a real implementation, you would need to set up a WhatsApp Business account
+        # and use their official API
+        
+        # Example using WhatsApp Business API
+        url = f"https://graph.facebook.com/v17.0/{whatsapp_phone_id}/messages"
+        headers = {
+            "Authorization": f"Bearer {whatsapp_api_key}",
+            "Content-Type": "application/json"
+        }
+        data = {
+            "messaging_product": "whatsapp",
+            "to": to_number,
+            "type": "text",
+            "text": {
+                "body": message
+            }
+        }
+        
+        # For demonstration, we'll just log the request instead of actually sending it
+        print(f"Would send WhatsApp message to {to_number}: {message}")
+        
+        # In a real implementation, you would uncomment this:
+        # response = requests.post(url, headers=headers, json=data)
+        # return response.status_code == 200
+        
+        return True
+    except Exception as e:
+        st.warning(f"WhatsApp notification failed: {str(e)}")
+        return False
+
+# Helper functions for notifications
+def validate_email(email):
+    """Validate email format"""
+    pattern = r'^[\w\.-]+@[\w\.-]+\.\w+$'
+    return re.match(pattern, email) is not None
+
+def validate_phone(phone):
+    """Validate phone number format (should include country code)"""
+    pattern = r'^\+\d{1,3}\d{6,14}$'
+    return re.match(pattern, phone) is not None
+
+def send_notification(notification_type, contact, message, image=None):
+    """Send notification based on type"""
+    if notification_type == "email":
+        if validate_email(contact):
+            return send_email_notification(
+                contact, 
+                "Anomaly Detected - Video Anomaly Detector", 
+                message,
+                image
+            )
+        else:
+            st.warning("Invalid email format. Notification not sent.")
+            return False
+    elif notification_type == "whatsapp":
+        if validate_phone(contact):
+            return send_whatsapp_notification(contact, message)
+        else:
+            st.warning("Invalid phone number format. Please include country code (e.g., +1234567890). Notification not sent.")
+            return False
+    return False
+
+# Helper functions for displaying results
+def display_single_result(result):
+    """Display a single analysis result"""
+    if isinstance(result, dict):
+        # This is a single frame result or cumulative result
+        if "anomaly_detected" in result:
+            # Create columns for image and text
+            if "frame" in result:
+                col1, col2 = st.columns([1, 2])
+                with col1:
+                    st.image(result["frame"], caption="Captured Frame", use_column_width=True)
+                
+                with col2:
+                    anomaly_detected = result["anomaly_detected"]
+                    
+                    # Start building the HTML content
+                    html_content = f"""
+                    <div class='result-details'>
+                    """
+                    
+                    # Add confidence if available
+                    if "confidence" in result:
+                        html_content += f"<p><strong>Confidence:</strong> {result['confidence']}%</p>"
+                    
+                    # Add analysis/text if available (check multiple possible keys)
+                    analysis_text = None
+                    for key in ["analysis", "text", "description"]:
+                        if key in result and result[key]:
+                            analysis_text = result[key]
+                            break
+                    
+                    if analysis_text:
+                        html_content += f"<p><strong>Analysis:</strong> {analysis_text}</p>"
+                    
+                    # Add anomaly type if available
+                    if "anomaly_type" in result and result["anomaly_type"]:
+                        html_content += f"<p><strong>Anomaly Type:</strong> {result['anomaly_type']}</p>"
+                    
+                    # Close the div
+                    html_content += "</div>"
+                    
+                    # Display the HTML content
+                    st.markdown(html_content, unsafe_allow_html=True)
+            else:
+                # No frame available, just show the text
+                # Start building the HTML content
+                html_content = "<div class='result-details'>"
+                
+                # Add confidence if available
+                if "confidence" in result:
+                    html_content += f"<p><strong>Confidence:</strong> {result['confidence']}%</p>"
+                
+                # Add analysis/text if available (check multiple possible keys)
+                analysis_text = None
+                for key in ["analysis", "text", "description"]:
+                    if key in result and result[key]:
+                        analysis_text = result[key]
+                        break
+                
+                if analysis_text:
+                    html_content += f"<p><strong>Analysis:</strong> {analysis_text}</p>"
+                
+                # Add anomaly type if available
+                if "anomaly_type" in result and result["anomaly_type"]:
+                    html_content += f"<p><strong>Anomaly Type:</strong> {result['anomaly_type']}</p>"
+                
+                # Close the div
+                html_content += "</div>"
+                
+                # Display the HTML content
+                st.markdown(html_content, unsafe_allow_html=True)
+        else:
+            # Display other types of results
+            st.json(result)
+    else:
+        # Unknown result type
+        st.write(result)
+
+def display_results(results, analysis_depth):
+    """Display analysis results based on analysis depth"""
+    if not results:
+        st.warning("No results to display")
+        return
+        
+    # Add a main results header
+    st.markdown("<h2 class='section-header'>📊 Analysis Results</h2>", unsafe_allow_html=True)
+    
+    # Add high-level summary at the top
+    if analysis_depth == "granular":
+        # For granular analysis, check if any frame has an anomaly
+        anomaly_frames = sum(1 for r in results if r.get("anomaly_detected", False))
+        total_frames = len(results)
+        
+        if anomaly_frames > 0:
+            # Get the anomaly types from frames with anomalies
+            anomaly_types = set(r.get("anomaly_type", "Unknown") for r in results if r.get("anomaly_detected", False))
+            anomaly_types_str = ", ".join(anomaly_types)
+            
+            st.markdown(
+                f"""
+                <div class='result-box anomaly'>
+                <h3>⚠️ ANOMALY DETECTED</h3>
+                <p><strong>Frames with anomalies:</strong> {anomaly_frames} out of {total_frames}</p>
+                <p><strong>Anomaly types:</strong> {anomaly_types_str}</p>
+                </div>
+                """, 
+                unsafe_allow_html=True
+            )
+        else:
+            st.markdown(
+                """
+                <div class='result-box normal'>
+                <h3>✅ No Anomalies Detected</h3>
+                <p>No anomalies were detected in any of the analyzed frames.</p>
+                </div>
+                """, 
+                unsafe_allow_html=True
+            )
+    else:  # cumulative
+        # For cumulative analysis, check the overall result
+        if results.get("anomaly_detected", False):
+            anomaly_type = results.get("anomaly_type", "Unknown")
+            st.markdown(
+                f"""
+                <div class='result-box anomaly'>
+                <h3>⚠️ ANOMALY DETECTED</h3>
+                <p><strong>Anomaly type:</strong> {anomaly_type}</p>
+                </div>
+                """, 
+                unsafe_allow_html=True
+            )
+        else:
+            st.markdown(
+                """
+                <div class='result-box normal'>
+                <h3>✅ No Anomalies Detected</h3>
+                <p>No anomalies were detected in the video.</p>
+                </div>
+                """, 
+                unsafe_allow_html=True
+            )
+        
+    # Display detailed results
+    if analysis_depth == "granular":
+        # For granular analysis, results is a list of frame analyses
+        st.markdown("<h3 class='sub-header'>🔍 Frame-by-Frame Analysis</h3>", unsafe_allow_html=True)
+        
+        # Display detailed view directly without tabs
+        for i, result in enumerate(results):
+            with st.expander(f"Frame {i+1} - {'⚠️ ANOMALY' if result.get('anomaly_detected', False) else '✅ Normal'}"):
+                display_single_result(result)
+    
+    else:  # cumulative
+        st.markdown("<h3 class='sub-header'>🔍 Overall Video Analysis</h3>", unsafe_allow_html=True)
+        display_single_result(results)
+        
+        # Display key frames if available
+        if "frames" in results and results["frames"]:
+            st.markdown("<h3 class='sub-header'>🖼️ Key Frames</h3>", unsafe_allow_html=True)
+            
+            # Create a row of columns for the frames
+            num_frames = len(results["frames"])
+            cols = st.columns(min(3, num_frames))
+            
+            # Display each frame in a column
+            for i, (col, frame) in enumerate(zip(cols, results["frames"])):
+                with col:
+                    st.image(frame, caption=f"Key Frame {i+1}", use_column_width=True)
+
+# Initialize session state for stop button
+if 'stop_requested' not in st.session_state:
+    st.session_state.stop_requested = False
+
+def request_stop():
+    st.session_state.stop_requested = True
+
+# Conditionally import Phi-4 detector
+try:
+    from phi4_detector import Phi4AnomalyDetector
+    PHI4_AVAILABLE = True
+except ImportError:
+    PHI4_AVAILABLE = False
+
+# Load environment variables from .env file
+load_dotenv()
+
+# Set page configuration
+st.set_page_config(
+    page_title="Video Anomaly Detector",
+    page_icon="🔍",
+    layout="wide"
+)
+
+# Custom CSS for better UI
+st.markdown("""
+<style>
+    @import url('https://fonts.googleapis.com/css2?family=Poppins:wght@300;400;500;600;700&display=swap');
+    
+    html, body, [class*="css"] {
+        font-family: 'Poppins', sans-serif;
+    }
+    
+    .main-header {
+        font-size: 2.8rem;
+        font-weight: 700;
+        color: #5046E5;
+        text-align: center;
+        margin-bottom: 1rem;
+        padding-top: 1.5rem;
+    }
+    
+    .sub-header {
+        font-size: 1.8rem;
+        font-weight: 600;
+        color: #36B37E;
+        margin-bottom: 1.2rem;
+    }
+    
+    .section-header {
+        font-size: 2rem;
+        font-weight: 600;
+        color: #5046E5;
+        margin-top: 2rem;
+        margin-bottom: 1rem;
+    }
+    
+    .result-box {
+        padding: 15px;
+        border-radius: 10px;
+        margin-bottom: 15px;
+    }
+    
+    .result-box.anomaly {
+        background-color: rgba(255, 76, 76, 0.1);
+        border: 1px solid rgba(255, 76, 76, 0.3);
+    }
+    
+    .result-box.normal {
+        background-color: rgba(54, 179, 126, 0.1);
+        border: 1px solid rgba(54, 179, 126, 0.3);
+    }
+    
+    .result-box h3 {
+        margin-top: 0;
+        margin-bottom: 10px;
+    }
+    
+    .result-box.anomaly h3 {
+        color: #FF4C4C;
+    }
+    
+    .result-box.normal h3 {
+        color: #36B37E;
+    }
+    
+    .result-container {
+        background-color: #f8f9fa;
+        padding: 1.8rem;
+        border-radius: 12px;
+        margin-bottom: 1.5rem;
+        border: 1px solid #e9ecef;
+        box-shadow: 0 4px 6px rgba(0, 0, 0, 0.05);
+    }
+    
+    .stProgress > div > div > div {
+        background-color: #5046E5;
+    }
+    
+    .stButton>button {
+        background-color: #5046E5;
+        color: white;
+        font-weight: 600;
+        border-radius: 8px;
+        padding: 0.5rem 1rem;
+        border: none;
+    }
+    
+    .stButton>button:hover {
+        background-color: #4038C7;
+    }
+    
+    .stSelectbox>div>div {
+        background-color: #f8f9fa;
+        border-radius: 8px;
+    }
+    
+    .stRadio>div {
+        padding: 10px;
+        background-color: #f8f9fa;
+        border-radius: 8px;
+    }
+    
+    .stExpander>div {
+        border-radius: 8px;
+        border: 1px solid #e9ecef;
+    }
+    
+    .model-info {
+        font-size: 0.9rem;
+        color: #6c757d;
+        font-style: italic;
+        margin-top: 0.5rem;
+    }
+    
+    .icon-text {
+        display: flex;
+        align-items: center;
+        gap: 0.5rem;
+    }
+    
+    .footer {
+        text-align: center;
+        color: #6c757d;
+        font-size: 0.9rem;
+        margin-top: 2rem;
+    }
+    
+    .anomaly-true {
+        color: #dc3545;
+        font-weight: bold;
+    }
+    
+    .anomaly-false {
+        color: #28a745;
+        font-weight: bold;
+    }
+    
+    .anomaly-type {
+        font-weight: bold;
+        margin-top: 0.5rem;
+    }
+    
+    .anomaly-box {
+        padding: 1rem;
+        border-radius: 8px;
+        margin-bottom: 1rem;
+    }
+    
+    .anomaly-box-true {
+        background-color: rgba(220, 53, 69, 0.1);
+        border: 1px solid rgba(220, 53, 69, 0.3);
+    }
+    
+    .anomaly-box-false {
+        background-color: rgba(40, 167, 69, 0.1);
+        border: 1px solid rgba(40, 167, 69, 0.3);
+    }
+    
+    .instructions-container {
+        font-size: 1.1rem;
+        line-height: 1.8;
+    }
+    
+    .instructions-container ol {
+        padding-left: 1.5rem;
+    }
+    
+    .instructions-container ul {
+        padding-left: 1.5rem;
+    }
+    
+    .instructions-container li {
+        margin-bottom: 0.5rem;
+    }
+    
+    .live-stream-container {
+        border: 2px solid #5046E5;
+        border-radius: 12px;
+        padding: 1rem;
+        margin-top: 1rem;
+    }
+    
+    .result-details {
+        padding: 15px;
+        border-radius: 10px;
+        margin-bottom: 15px;
+        background-color: rgba(80, 70, 229, 0.05);
+        border: 1px solid rgba(80, 70, 229, 0.2);
+    }
+    
+    .result-details p {
+        margin-bottom: 10px;
+    }
+    
+    .result-details strong {
+        color: #5046E5;
+    }
+    
+    .video-preview-container {
+        border: 1px solid #e9ecef;
+        border-radius: 10px;
+        padding: 15px;
+        margin-bottom: 20px;
+        background-color: rgba(80, 70, 229, 0.03);
+    }
+    
+    .video-preview-container video {
+        width: 100%;
+        border-radius: 8px;
+        margin-bottom: 10px;
+    }
+    
+    .video-info {
+        display: flex;
+        justify-content: space-between;
+        margin-top: 10px;
+    }
+    
+    .video-info-item {
+        text-align: center;
+        padding: 8px;
+        background-color: #f8f9fa;
+        border-radius: 5px;
+        flex: 1;
+        margin: 0 5px;
+    }
+</style>
+""", unsafe_allow_html=True)
+
+# Header with icon
+st.markdown("<h1 class='main-header'>🔍 Video Anomaly Detector</h1>", unsafe_allow_html=True)
+st.markdown("<p style='text-align: center; font-size: 1.2rem; margin-bottom: 2rem;'>Analyze video frames for anomalies using advanced AI models</p>", unsafe_allow_html=True)
+
+# Sidebar for inputs
+with st.sidebar:
+    st.markdown("<h2 class='sub-header'>⚙️ Settings</h2>", unsafe_allow_html=True)
+    
+    # Input source selection
+    st.markdown("<div class='icon-text'><span>📹</span><span>Input Source</span></div>", unsafe_allow_html=True)
+    input_source = st.radio(
+        "",
+        ["Video File", "Live Stream"],
+        index=0,
+        help="Select the input source for analysis"
+    )
+    
+    # File uploader or stream URL based on input source
+    if input_source == "Video File":
+        st.markdown("<div class='icon-text'><span>📁</span><span>Upload Video</span></div>", unsafe_allow_html=True)
+        
+        # Find sample .mp4 files in the current directory
+        sample_files = []
+        for file in os.listdir():
+            if file.endswith('.mp4'):
+                sample_files.append(file)
+        
+        # Show sample files if available
+        if sample_files:
+            st.info(f"Sample videos available: {', '.join(sample_files)}")
+            use_sample = st.checkbox("Use a sample video instead of uploading")
+            
+            if use_sample:
+                selected_sample = st.selectbox("Select a sample video", sample_files)
+                uploaded_file = selected_sample  # We'll handle this specially later
+                
+                # Add video preview section
+                st.markdown("<h3 class='sub-header'>🎬 Video Preview</h3>", unsafe_allow_html=True)
+                
+                # Create a container for the video preview with custom styling
+                st.markdown("<div class='video-preview-container'>", unsafe_allow_html=True)
+                
+                # Get the full path to the selected sample video
+                video_path = os.path.join(os.getcwd(), selected_sample)
+                
+                # Display the video player
+                st.video(video_path)
+                
+                # Display video information
+                try:
+                    cap = cv2.VideoCapture(video_path)
+                    if cap.isOpened():
+                        # Get video properties
+                        fps = cap.get(cv2.CAP_PROP_FPS)
+                        frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+                        width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+                        height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+                        
+                        # Calculate duration
+                        duration = frame_count / fps if fps > 0 else 0
+                        
+                        # Format duration as minutes:seconds
+                        minutes = int(duration // 60)
+                        seconds = int(duration % 60)
+                        duration_str = f"{minutes}:{seconds:02d}"
+                        
+                        
+                        cap.release()
+                except Exception as e:
+                    st.warning(f"Could not read video properties: {str(e)}")
+                
+                st.markdown("</div>", unsafe_allow_html=True)
+            else:
+                uploaded_file = st.file_uploader("", type=["mp4", "avi", "mov", "mkv"])
+        else:
+            uploaded_file = st.file_uploader("", type=["mp4", "avi", "mov", "mkv"])
+        
+        stream_source = None
+    else:  # Live Stream
+        st.markdown("<div class='icon-text'><span>🔗</span><span>Stream Source</span></div>", unsafe_allow_html=True)
+        stream_options = ["Webcam", "IP Camera / RTSP Stream"]
+        stream_type = st.selectbox("", stream_options, index=0)
+        
+        if stream_type == "Webcam":
+            stream_source = 0  # Default webcam
+        else:
+            stream_source = st.text_input("Stream URL", placeholder="rtsp://username:password@ip_address:port/path")
+        
+        # Max frames to process for live stream
+        st.markdown("<div class='icon-text'><span>🔢</span><span>Frame Capture Settings</span></div>", unsafe_allow_html=True)
+        
+        capture_mode = st.radio(
+            "Capture Mode",
+            ["Frame Count Limit", "Time Interval (Continuous)"],
+            index=0,
+            help="Choose how to capture frames from the live stream"
+        )
+        
+        if capture_mode == "Frame Count Limit":
+            max_frames = st.number_input(
+                "Maximum Frames",
+                min_value=1,
+                max_value=100,
+                value=30,
+                help="Maximum number of frames to process from the live stream"
+            )
+            time_interval = None
+        else:  # Time Interval mode
+            max_frames = None  # No frame limit in time interval mode
+            time_interval = st.number_input(
+                "Seconds Between Captures",
+                min_value=1,
+                max_value=60,
+                value=5,
+                help="Capture one frame every X seconds indefinitely"
+            )
+            st.info("⚠️ In time interval mode, processing will continue indefinitely. Use the Stop button to end capture.")
+        
+        uploaded_file = None
+    
+    # Model selection
+    st.markdown("<div class='icon-text'><span>🧠</span><span>AI Model</span></div>", unsafe_allow_html=True)
+    
+    # Add Phi-4 to the model options if available
+    model_options = ["GPT-4o", "GPT-4o-mini"]
+    if PHI4_AVAILABLE:
+        model_options.append("Phi-4")
+    model_options.append("Phi-3 (Coming Soon)")
+    
+    model = st.selectbox(
+        "",
+        model_options,
+        index=0,
+        help="Select the AI model to use for analysis"
+    )
+    
+    # Display model info based on selection
+    if model == "GPT-4o":
+        st.markdown("<div class='model-info'>Most powerful model with highest accuracy</div>", unsafe_allow_html=True)
+        model_value = "gpt-4o"
+        use_phi4 = False
+    elif model == "GPT-4o-mini":
+        st.markdown("<div class='model-info'>Faster and more cost-effective</div>", unsafe_allow_html=True)
+        model_value = "gpt-4o-mini"
+        use_phi4 = False
+    elif model == "Phi-4":
+        st.markdown("<div class='model-info'>Microsoft's multimodal model, runs locally</div>", unsafe_allow_html=True)
+        model_value = "phi-4"
+        use_phi4 = True
+    else:  # Phi-3
+        st.markdown("<div class='model-info'>Not yet implemented</div>", unsafe_allow_html=True)
+        model_value = "gpt-4o"  # Default to GPT-4o if Phi-3 is selected
+        use_phi4 = False
+        st.warning("Phi-3 support is coming soon. Using GPT-4o instead.")
+    
+    # Skip frames input with icon
+    st.markdown("<div class='icon-text'><span>⏭️</span><span>Frame Skip Rate</span></div>", unsafe_allow_html=True)
+    skip_frames = st.number_input(
+        "", 
+        min_value=0, 
+        max_value=100, 
+        value=5,
+        help="Higher values process fewer frames, making analysis faster but potentially less accurate"
+    )
+    
+    # Analysis depth selection
+    st.markdown("<div class='icon-text'><span>🔬</span><span>Analysis Depth</span></div>", unsafe_allow_html=True)
+    analysis_depth = st.radio(
+        "",
+        ["Granular (Frame by Frame)", "Cumulative (Overall)"],
+        index=0,
+        help="Granular provides analysis for each frame, Cumulative gives an overall assessment"
+    )
+    
+    # Map the radio button value to the actual value
+    analysis_depth_value = "granular" if analysis_depth == "Granular (Frame by Frame)" else "cumulative"
+    
+    # Notification options
+    st.markdown("<div class='icon-text'><span>🔔</span><span>Notifications</span></div>", unsafe_allow_html=True)
+    enable_notifications = st.checkbox("Enable notifications for anomaly detection", value=False)
+    
+    if enable_notifications:
+        notification_type = st.radio(
+            "Notification Method",
+            ["Email", "WhatsApp"],
+            index=0,
+            help="Select how you want to be notified when anomalies are detected"
+        )
+        
+        if notification_type == "Email":
+            notification_email = st.text_input(
+                "Email Address",
+                placeholder="[email protected]",
+                help="Enter the email address to receive notifications"
+            )
+            st.session_state.notification_contact = notification_email if notification_email else None
+            st.session_state.notification_type = "email" if notification_email else None
+            
+        else:  # WhatsApp
+            notification_phone = st.text_input(
+                "WhatsApp Number",
+                placeholder="+1234567890 (include country code)",
+                help="Enter your WhatsApp number with country code"
+            )
+            st.session_state.notification_contact = notification_phone if notification_phone else None
+            st.session_state.notification_type = "whatsapp" if notification_phone else None
+    else:
+        st.session_state.notification_type = None
+        st.session_state.notification_contact = None
+    
+    # Prompt input with icon
+    st.markdown("<div class='icon-text'><span>💬</span><span>Anomaly Description</span></div>", unsafe_allow_html=True)
+    prompt = st.text_area(
+        "", 
+        value="Analyze this frame and describe if there are any unusual or anomalous activities or objects. If you detect anything unusual, explain what it is and why it might be considered an anomaly.",
+        height=150,
+        help="Describe what kind of anomaly to look for"
+    )
+    
+    # API key input with default from environment variable and icon (only show for OpenAI models)
+    if not use_phi4:
+        st.markdown("<div class='icon-text'><span>🔑</span><span>OpenAI API Key</span></div>", unsafe_allow_html=True)
+        default_api_key = os.getenv("OPENAI_API_KEY", "")
+        api_key = st.text_input(
+            "", 
+            value=default_api_key,
+            type="password", 
+            help="Your OpenAI API key with access to the selected model"
+        )
+    else:
+        # For Phi-4, we don't need an API key
+        api_key = "not-needed-for-phi4"
+    
+    # Submit button with icon
+    submit_button = st.button("���� Analyze Video")
+
+# Main content area for video file
+if input_source == "Video File" and uploaded_file is not None:
+    # Display video info
+    st.markdown("<h2 class='sub-header'>📊 Video Information</h2>", unsafe_allow_html=True)
+    
+    # Check if we're using a sample file or an uploaded file
+    if isinstance(uploaded_file, str) and os.path.exists(uploaded_file):
+        # This is a sample file from the directory
+        video_path = uploaded_file
+        st.success(f"Using sample video: {os.path.basename(video_path)}")
+    else:
+        # This is an uploaded file
+        # Save uploaded file to a temporary file
+        with tempfile.NamedTemporaryFile(delete=False, suffix='.mp4') as tmp_file:
+            tmp_file.write(uploaded_file.getvalue())
+            video_path = tmp_file.name
+    
+    # Get video metadata
+    # For video files, use the default backend instead of DirectShow
+    cap = cv2.VideoCapture(video_path)
+    
+    # Don't set MJPG format for video files as it can interfere with proper decoding
+    # cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M','J','P','G'))
+    
+    # Try to get video properties
+    fps = cap.get(cv2.CAP_PROP_FPS)
+    frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+    
+    # Prevent division by zero, but only show warning for live streams
+    # For video files, this is likely an actual error
+    if fps <= 0:
+        # Check if this is a video file (not a webcam/stream)
+        if isinstance(video_path, str) and os.path.exists(video_path):
+            # This is a file that exists but has FPS issues
+            fps = 30.0  # Use a default value
+            st.warning(f"Could not determine frame rate for video file: {os.path.basename(video_path)}. Using default value of 30 FPS.")
+        else:
+            # This is likely a webcam or stream
+            fps = 30.0
+            st.info("Using default frame rate of 30 FPS for live stream.")
+    
+    duration = frame_count / fps
+    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+    cap.release()
+    
+    # Display video metadata in a nicer format
+    col1, col2, col3 = st.columns(3)
+    with col1:
+        st.markdown("<div style='text-align: center;'>⏱️</div>", unsafe_allow_html=True)
+        st.metric("Duration", f"{duration:.2f} seconds")
+    with col2:
+        st.markdown("<div style='text-align: center;'>🎞️</div>", unsafe_allow_html=True)
+        st.metric("Total Frames", frame_count)
+    with col3:
+        st.markdown("<div style='text-align: center;'>📐</div>", unsafe_allow_html=True)
+        st.metric("Resolution", f"{width}x{height}")
+    
+    # Display estimated frames to process
+    estimated_frames = frame_count // (skip_frames + 1) + 1
+    st.info(f"With current settings, approximately {estimated_frames} frames will be processed.")
+
+# Main content area for live stream
+elif input_source == "Live Stream" and stream_source is not None:
+    # Display live stream info
+    st.markdown("<h2 class='sub-header'>📊 Live Stream Information</h2>", unsafe_allow_html=True)
+    
+    # Display stream source info
+    if stream_source == 0:
+        st.info("Using default webcam as the stream source.")
+    else:
+        st.info(f"Using stream URL: {stream_source}")
+    
+    # Display estimated frames to process
+    st.info(f"Will process up to {max_frames} frames with a skip rate of {skip_frames}.")
+    
+    # Show a placeholder for the live stream
+    st.markdown("<div class='live-stream-container'><p style='text-align: center;'>Live stream preview will appear here during processing</p></div>", unsafe_allow_html=True)
+
+# Process video or stream when submit button is clicked
+if submit_button:
+    if not api_key and not use_phi4:
+        st.error("⚠️ Please enter your OpenAI API key")
+    elif input_source == "Video File" and uploaded_file is None:
+        st.error("⚠️ Please upload a video file")
+    elif input_source == "Live Stream" and stream_source is None:
+        st.error("⚠️ Please provide a valid stream source")
+    else:
+        try:
+            # Initialize detector based on selected model
+            if use_phi4:
+                with st.spinner("Loading Phi-4 model... This may take a while if downloading for the first time."):
+                    detector = Phi4AnomalyDetector()
+                st.success("Phi-4 model loaded successfully!")
+            else:
+                detector = VideoAnomalyDetector(api_key, model_value)
+            
+            # Progress bar and status
+            st.markdown("<h2 class='sub-header'>⏳ Processing Video</h2>", unsafe_allow_html=True)
+            progress_bar = st.progress(0)
+            status_text = st.empty()
+            
+            # Create a callback function to update progress
+            def update_progress(current, total):
+                if total == -1:
+                    # Continuous mode
+                    status_text.text(f"Processed {current} frames (continuous mode)...")
+                else:
+                    # Normal mode with a known total
+                    if total > 0:
+                        progress = current / total
+                        progress_bar.progress(progress)
+                    else:
+                        # Handle case where total is zero
+                        progress_bar.progress(0)
+                    status_text.text(f"Processing frame {current+1} of {total if total > 0 else '?'}...")
+            
+            # Process the video or stream
+            start_time = time.time()
+            
+            if input_source == "Video File":
+                results = detector.process_video(video_path, skip_frames, prompt, analysis_depth_value, update_progress)
+                print(f"Results: {results}")
+                # Results will be displayed after processing
+                
+            else:  # Live Stream
+                if capture_mode == "Frame Count Limit":
+                    # Process with frame count limit (original behavior)
+                    results = detector.process_live_stream(stream_source, skip_frames, prompt, analysis_depth_value, max_frames, update_progress)
+                    # Results will be displayed after processing
+                    
+                else:  # Time Interval mode
+                    # Create a placeholder for continuous results
+                    results_container = st.empty()
+                    
+                    # Reset stop request flag at the beginning of processing
+                    st.session_state.stop_requested = False
+                    
+                    # Create a stop button outside the loop
+                    st.button("Stop Capture", key="stop_continuous_main", on_click=request_stop)
+                    
+                    # Process with time interval (generator mode)
+                    results_generator = detector.process_live_stream(
+                        stream_source, skip_frames, prompt, analysis_depth_value, 
+                        None, update_progress, time_interval
+                    )
+                    
+                    # Collect results for cumulative analysis if needed
+                    all_results = []
+                    frame_counter = 0
+                    
+                    try:
+                        # Process results as they come in
+                        for result in results_generator:
+                            # Check if stop button was pressed
+                            if st.session_state.stop_requested:
+                                st.success("Capture stopped by user")
+                                break
+                                
+                            frame_counter += 1
+                            all_results.append(result)
+                            
+                            # Display the latest result
+                            with results_container.container():
+                                if analysis_depth_value == "granular":
+                                    # For granular analysis, show the latest frame result
+                                    st.markdown(f"### Frame {frame_counter}")
+                                    display_single_result(result)
+                                    
+                                    # Send notification if anomaly detected and notifications are enabled
+                                    if result.get("anomaly_detected", False) and st.session_state.notification_type and st.session_state.notification_contact:
+                                        # Create notification message
+                                        anomaly_type = result.get("anomaly_type", "Unknown")
+                                        anomaly_message = f"Anomaly detected in live stream (Frame {frame_counter}).\n"
+                                        anomaly_message += f"Anomaly type: {anomaly_type}\n\n"
+                                        
+                                        # Add analysis details
+                                        analysis_text = None
+                                        for key in ["analysis", "text", "description"]:
+                                            if key in result and result[key]:
+                                                analysis_text = result[key]
+                                                break
+                                        
+                                        if analysis_text:
+                                            anomaly_message += f"Analysis: {analysis_text[:500]}..."
+                                        
+                                        # Send notification
+                                        with st.spinner("Sending notification about detected anomaly..."):
+                                            notification_sent = send_notification(
+                                                st.session_state.notification_type,
+                                                st.session_state.notification_contact,
+                                                anomaly_message,
+                                                result.get("frame")
+                                            )
+                                            
+                                            if notification_sent:
+                                                st.success(f"Notification sent to {st.session_state.notification_contact} via {st.session_state.notification_type.capitalize()}")
+                                            else:
+                                                st.error(f"Failed to send notification. Please check your {st.session_state.notification_type} settings.")
+                                else:
+                                    # For cumulative analysis, we get periodic updates
+                                    st.markdown(f"### Cumulative Analysis (Updated)")
+                                    display_single_result(result)
+                                    
+                                    # Send notification if anomaly detected and notifications are enabled
+                                    if result.get("anomaly_detected", False) and st.session_state.notification_type and st.session_state.notification_contact:
+                                        # Create notification message
+                                        anomaly_type = result.get("anomaly_type", "Unknown")
+                                        anomaly_message = f"Anomaly detected in live stream (Cumulative Analysis).\n"
+                                        anomaly_message += f"Anomaly type: {anomaly_type}\n\n"
+                                        
+                                        # Add analysis details
+                                        analysis_text = None
+                                        for key in ["analysis", "text", "description"]:
+                                            if key in result and result[key]:
+                                                analysis_text = result[key]
+                                                break
+                                        
+                                        if analysis_text:
+                                            anomaly_message += f"Analysis: {analysis_text[:500]}..."
+                                        
+                                        # Get a frame for the notification if available
+                                        anomaly_image = None
+                                        if "frames" in result and result["frames"]:
+                                            anomaly_image = result["frames"][0]
+                                        
+                                        # Send notification
+                                        with st.spinner("Sending notification about detected anomaly..."):
+                                            notification_sent = send_notification(
+                                                st.session_state.notification_type,
+                                                st.session_state.notification_contact,
+                                                anomaly_message,
+                                                anomaly_image
+                                            )
+                                            
+                                            if notification_sent:
+                                                st.success(f"Notification sent to {st.session_state.notification_contact} via {st.session_state.notification_type.capitalize()}")
+                                            else:
+                                                st.error(f"Failed to send notification. Please check your {st.session_state.notification_type} settings.")
+                            
+                            # Sleep briefly to allow UI updates
+                            time.sleep(0.1)
+                    except StopIteration:
+                        if not st.session_state.stop_requested:
+                            st.info("Stream ended")
+                    
+                    # Final results
+                    if analysis_depth_value == "granular":
+                        results = all_results
+                    else:
+                        results = all_results[-1] if all_results else None
+            
+            end_time = time.time()
+            
+            # Calculate processing time
+            processing_time = end_time - start_time
+            st.success(f"Processing completed in {processing_time:.2f} seconds")
+            
+            # Check if notifications are enabled and if anomalies were detected
+            if st.session_state.notification_type and st.session_state.notification_contact:
+                # Check if anomalies were detected
+                anomalies_detected = False
+                anomaly_image = None
+                anomaly_message = ""
+                
+                if analysis_depth_value == "granular":
+                    # For granular analysis, check if any frame has an anomaly
+                    anomaly_frames = [r for r in results if r.get("anomaly_detected", False)]
+                    if anomaly_frames:
+                        anomalies_detected = True
+                        # Get the first anomaly frame for the notification
+                        first_anomaly = anomaly_frames[0]
+                        anomaly_image = first_anomaly.get("frame")
+                        
+                        # Create notification message
+                        anomaly_types = set(r.get("anomaly_type", "Unknown") for r in anomaly_frames)
+                        anomaly_message = f"Anomaly detected in {len(anomaly_frames)} out of {len(results)} frames.\n"
+                        anomaly_message += f"Anomaly types: {', '.join(anomaly_types)}\n\n"
+                        
+                        # Add details of the first anomaly
+                        analysis_text = None
+                        for key in ["analysis", "text", "description"]:
+                            if key in first_anomaly and first_anomaly[key]:
+                                analysis_text = first_anomaly[key]
+                                break
+                        
+                        if analysis_text:
+                            anomaly_message += f"Analysis of first anomaly: {analysis_text[:500]}..."
+                else:
+                    # For cumulative analysis, check the overall result
+                    if results.get("anomaly_detected", False):
+                        anomalies_detected = True
+                        
+                        # Get a frame for the notification if available
+                        if "frames" in results and results["frames"]:
+                            anomaly_image = results["frames"][0]
+                        
+                        # Create notification message
+                        anomaly_type = results.get("anomaly_type", "Unknown")
+                        anomaly_message = f"Anomaly detected in video analysis.\n"
+                        anomaly_message += f"Anomaly type: {anomaly_type}\n\n"
+                        
+                        # Add analysis details
+                        analysis_text = None
+                        for key in ["analysis", "text", "description"]:
+                            if key in results and results[key]:
+                                analysis_text = results[key]
+                                break
+                        
+                        if analysis_text:
+                            anomaly_message += f"Analysis: {analysis_text[:500]}..."
+                
+                # Send notification if anomalies were detected
+                if anomalies_detected:
+                    with st.spinner("Sending notification about detected anomalies..."):
+                        notification_sent = send_notification(
+                            st.session_state.notification_type,
+                            st.session_state.notification_contact,
+                            anomaly_message,
+                            anomaly_image
+                        )
+                        
+                        if notification_sent:
+                            st.success(f"Notification sent to {st.session_state.notification_contact} via {st.session_state.notification_type.capitalize()}")
+                        else:
+                            st.error(f"Failed to send notification. Please check your {st.session_state.notification_type} settings.")
+            
+            # Only display results here if we're not in time interval mode
+            # (time interval mode displays results as they come in)
+            if not (input_source == "Live Stream" and capture_mode == "Time Interval (Continuous)"):
+                # Display the results without an additional header
+                display_results(results, analysis_depth_value)
+                
+            # Download results button
+            if results:
+                try:
+                    # Convert results to JSON using our custom encoder
+                    results_json = json.dumps(results, indent=2, cls=NumpyEncoder)
+                    
+                    # Create a download button
+                    st.download_button(
+                        label="Download Results as JSON",
+                        data=results_json,
+                        file_name="anomaly_detection_results.json",
+                        mime="application/json"
+                    )
+                except Exception as e:
+                    st.warning(f"Could not create downloadable results: {str(e)}")
+                    st.info("This is usually due to large image data in the results. The analysis is still valid.")
+            
+            # Clean up the temporary file if using a video file
+            if input_source == "Video File" and 'video_path' in locals():
+                # Only delete the file if it's a temporary file, not a sample file
+                if not isinstance(uploaded_file, str):
+                    os.unlink(video_path)
+            
+        except Exception as e:
+            st.error(f"⚠️ An error occurred: {str(e)}")
+            if input_source == "Video File" and 'video_path' in locals():
+                # Only delete the file if it's a temporary file, not a sample file
+                if not isinstance(uploaded_file, str):
+                    os.unlink(video_path)
+
+# Instructions when no file is uploaded or stream is selected
+if (input_source == "Video File" and uploaded_file is None) or (input_source == "Live Stream" and stream_source is None) or not submit_button:
+    # Using HTML component to properly render the HTML
+    model_options_html = ""
+    if PHI4_AVAILABLE:
+        model_options_html += "<li><strong>Phi-4</strong> - Microsoft's multimodal model, runs locally</li>"
+    
+    instructions_html = f"""
+    <div class="result-container instructions-container">
+        <h2 style="color: #5046E5;">📝 How to use this application</h2>
+        
+        <ol>
+            <li><strong>Select an input source</strong>:
+                <ul>
+                    <li><strong>Video File</strong> - Upload a video file for analysis</li>
+                    <li><strong>Live Stream</strong> - Connect to a webcam or IP camera stream</li>
+                </ul>
+            </li>
+            <li><strong>Select an AI model</strong> for analysis:
+                <ul>
+                   <li><strong>GPT-4o-mini</strong> - Faster and more cost-effective</li>
+                   <li><strong>GPT-4o</strong> - Most powerful model with highest accuracy</li>                  
+                    {model_options_html}
+                </ul>
+            </li>
+            <li><strong>Set the number of frames to skip</strong> - higher values process fewer frames</li>
+            <li><strong>Choose an analysis depth</strong>:
+                <ul>
+                    <li><strong>Granular</strong> - Analyzes each frame individually</li>
+                    <li><strong>Cumulative</strong> - Provides an overall summary with key frames</li>
+                </ul>
+            </li>
+            <li><strong>Enter a prompt</strong> describing what anomaly to look for</li>
+            <li><strong>Enter your OpenAI API key</strong> with access to the selected model (not needed for Phi-4)</li>
+            <li><strong>Click "Analyze Video"</strong> to start processing</li>
+        </ol>
+        
+        <p>The application will extract frames from your video or stream, analyze them using the selected AI model, and display the results with clear indicators for detected anomalies.</p>
+    </div>
+    """
+    components.html(instructions_html, height=500)
+
+# Footer
+st.markdown("---")
+st.markdown("<div class='footer'>Powered by OpenAI's GPT-4o, GPT-4o-mini, and Microsoft's Phi-4 models | © 2023 Video Anomaly Detector</div>", unsafe_allow_html=True)
+
+
+
+
+

detector.py

@@ -0,0 +1,465 @@
+import cv2
+import numpy as np
+import base64
+import os
+import tempfile
+from openai import OpenAI
+from PIL import Image
+import io
+import re
+import time
+
+class VideoAnomalyDetector:
+    def __init__(self, api_key, model="gpt-4o"):
+        """
+        Initialize the VideoAnomalyDetector with OpenAI API key.
+        
+        Args:
+            api_key (str): OpenAI API key for accessing GPT-4o model
+            model (str): Model to use for analysis ("gpt-4o" or "gpt-4o-mini")
+        """
+        self.client = OpenAI(api_key=api_key)
+        self.model = model
+        
+    def extract_frames(self, video_path, skip_frames):
+        """
+        Extract frames from a video file, skipping the specified number of frames.
+        
+        Args:
+            video_path (str): Path to the video file
+            skip_frames (int): Number of frames to skip between captures
+            
+        Returns:
+            list: List of extracted frames as numpy arrays
+        """
+        frames = []
+        # Use the default backend for video files
+        # DirectShow can cause issues with some video files
+        cap = cv2.VideoCapture(video_path)
+        
+        if not cap.isOpened():
+            raise ValueError(f"Could not open video file: {video_path}")
+        
+        # Don't set MJPG format for video files as it can interfere with proper decoding
+        # cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M','J','P','G'))
+        
+        frame_count = 0
+        while True:
+            ret, frame = cap.read()
+            if not ret:
+                break
+                
+            if frame_count % (skip_frames + 1) == 0:
+                # Convert from BGR to RGB
+                rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                frames.append(rgb_frame)
+                
+            frame_count += 1
+            
+        cap.release()
+        return frames
+    
+    def process_live_stream(self, stream_source, skip_frames, prompt, analysis_depth="granular", max_frames=None, callback=None, time_interval=None):
+        """
+        Process frames from a live video stream.
+        
+        Args:
+            stream_source: Stream source (0 for webcam, URL for IP camera or RTSP stream)
+            skip_frames (int): Number of frames to skip between captures
+            prompt (str): Prompt describing what anomaly to look for
+            analysis_depth (str): "granular" for frame-by-frame analysis or "cumulative" for overall analysis
+            max_frames (int, optional): Maximum number of frames to process (None for unlimited)
+            callback (function, optional): Callback function to report progress
+            time_interval (int, optional): If set, capture one frame every X seconds instead of using skip_frames
+            
+        Returns:
+            list or dict: List of analysis results for each processed frame (granular) or dict with cumulative analysis (cumulative)
+        """
+        # Open the video stream with appropriate backend
+        # Only use DirectShow for local webcams (0 or 1) on Windows
+        if os.name == 'nt' and (stream_source == 0 or stream_source == 1):
+            # This is a local webcam on Windows, use DirectShow
+            cap = cv2.VideoCapture(stream_source, cv2.CAP_DSHOW)
+            
+            # For webcams, MJPG format can be more stable
+            cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M','J','P','G'))
+        else:
+            # For IP cameras, RTSP streams, or non-Windows systems, use default backend
+            cap = cv2.VideoCapture(stream_source)
+        
+        if not cap.isOpened():
+            raise ValueError(f"Could not open video stream: {stream_source}")
+        
+        frames = []
+        frame_count = 0
+        processed_count = 0
+        last_capture_time = time.time()
+        
+        try:
+            while True:
+                ret, frame = cap.read()
+                if not ret:
+                    break
+                
+                current_time = time.time()
+                
+                # Determine if we should capture this frame
+                should_capture = False
+                
+                if time_interval is not None:
+                    # Time-based interval mode
+                    if current_time - last_capture_time >= time_interval:
+                        should_capture = True
+                        last_capture_time = current_time
+                else:
+                    # Frame-skip mode
+                    if frame_count % (skip_frames + 1) == 0:
+                        should_capture = True
+                
+                if should_capture:
+                    # Convert from BGR to RGB
+                    rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                    frames.append(rgb_frame)
+                    processed_count += 1
+                    
+                    # Process the frame immediately in time interval mode
+                    if time_interval is not None:
+                        # Process the frame immediately
+                        result = self.analyze_frame(rgb_frame, prompt)
+                        
+                        # Make sure the frame is included in the result
+                        if "frame" not in result:
+                            result["frame"] = rgb_frame
+                        
+                        # If we have a callback, call it with -1 for total to indicate continuous mode
+                        if callback:
+                            callback(processed_count, -1)
+                        
+                        # In time interval mode, we yield results one by one
+                        if analysis_depth == "granular":
+                            yield result
+                        else:
+                            # For cumulative analysis, we need to keep all frames
+                            # We'll handle this outside the loop
+                            pass
+                    else:
+                        # Update progress if callback is provided (frame count mode)
+                        if callback and max_frames:
+                            callback(processed_count, max_frames)
+                        
+                        # Break if we've reached the maximum number of frames
+                        if max_frames and processed_count >= max_frames:
+                            break
+                
+                frame_count += 1
+        finally:
+            cap.release()
+        
+        # If we're in time interval mode with cumulative analysis, we don't return here
+        # as we're yielding results above
+        if time_interval is not None and analysis_depth == "cumulative":
+            # This is a special case - we need to periodically do cumulative analysis
+            # For simplicity, we'll just return the current cumulative analysis
+            result = self.analyze_frames_cumulatively(frames, prompt, callback)
+            yield result
+            return
+            
+        # Process the collected frames for non-time interval mode
+        if time_interval is None:
+            if analysis_depth == "cumulative":
+                return self.analyze_frames_cumulatively(frames, prompt, callback)
+            else:  # granular (default)
+                results = []
+                
+                for i, frame in enumerate(frames):
+                    if callback:
+                        callback(i, len(frames))
+                        
+                    result = self.analyze_frame(frame, prompt)
+                    results.append(result)
+                    
+                return results
+    
+    def encode_image_to_base64(self, image_array):
+        """
+        Convert a numpy array image to base64 encoded string.
+        
+        Args:
+            image_array (numpy.ndarray): Image as numpy array
+            
+        Returns:
+            str: Base64 encoded image string
+        """
+        # Convert numpy array to PIL Image
+        pil_image = Image.fromarray(image_array)
+        
+        # Create a bytes buffer
+        buffer = io.BytesIO()
+        
+        # Save the image to the buffer in PNG format
+        pil_image.save(buffer, format="PNG")
+        
+        # Get the bytes from the buffer
+        img_bytes = buffer.getvalue()
+        
+        # Encode the bytes to base64
+        base64_encoded = base64.b64encode(img_bytes).decode('utf-8')
+        
+        return base64_encoded
+    
+    def analyze_frame(self, frame, prompt):
+        """
+        Analyze a frame using the selected OpenAI model.
+        
+        Args:
+            frame (numpy.ndarray): Frame to analyze
+            prompt (str): Prompt describing what anomaly to look for
+            
+        Returns:
+            dict: Analysis result from the model
+        """
+        base64_image = self.encode_image_to_base64(frame)
+        
+        # Enhanced prompt to get structured information about anomalies
+        enhanced_prompt = f"""
+{prompt}
+
+After your analysis, please include a structured assessment at the end of your response in this exact format:
+ANOMALY_DETECTED: [Yes/No]
+ANOMALY_TYPE: [Human/Non-human/None]
+CONFIDENCE: [0-100]
+
+For ANOMALY_DETECTED, answer "Yes" if you detect any anomaly, otherwise "No".
+For ANOMALY_TYPE, if an anomaly is detected, classify it as either "Human" (if it involves people or human activities) or "Non-human" (if it involves objects, animals, or environmental factors). If no anomaly is detected, use "None".
+For CONFIDENCE, provide a number from 0 to 100 indicating your confidence level in the assessment.
+"""
+        
+        try:
+            response = self.client.chat.completions.create(
+                model=self.model,
+                messages=[
+                    {
+                        "role": "user",
+                        "content": [
+                            {"type": "text", "text": enhanced_prompt},
+                            {
+                                "type": "image_url",
+                                "image_url": {
+                                    "url": f"data:image/png;base64,{base64_image}"
+                                }
+                            }
+                        ]
+                    }
+                ],
+                max_tokens=1000
+            )
+            
+            # Extract the response text
+            response_text = response.choices[0].message.content
+            
+            # Extract anomaly detection information using regex
+            anomaly_detected = False
+            anomaly_type = "None"
+            confidence = 0
+            
+            # Look for the structured format
+            anomaly_match = re.search(r'ANOMALY_DETECTED:\s*(Yes|No)', response_text, re.IGNORECASE)
+            if anomaly_match and anomaly_match.group(1).lower() == 'yes':
+                anomaly_detected = True
+                confidence = 90  # Default high confidence when anomaly is detected
+                
+                # If anomaly detected, look for the type
+                type_match = re.search(r'ANOMALY_TYPE:\s*(Human|Non-human|None)', response_text, re.IGNORECASE)
+                if type_match:
+                    anomaly_type = type_match.group(1)
+                    
+                # Look for confidence information
+                conf_match = re.search(r'CONFIDENCE:\s*(\d+)', response_text, re.IGNORECASE)
+                if conf_match:
+                    try:
+                        confidence = int(conf_match.group(1))
+                    except:
+                        pass  # Keep default confidence if parsing fails
+            
+            return {
+                "text": response_text,
+                "analysis": response_text,  # Add analysis field as an alias for text
+                "frame": frame,
+                "anomaly_detected": anomaly_detected,
+                "anomaly_type": anomaly_type,
+                "confidence": confidence,  # Add confidence field
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+        except Exception as e:
+            return {
+                "error": str(e),
+                "frame": frame,
+                "anomaly_detected": False,
+                "anomaly_type": "None",
+                "confidence": 0,  # Add confidence field
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+    
+    def analyze_frames_cumulatively(self, frames, prompt, callback=None):
+        """
+        Analyze all frames together and provide a cumulative analysis.
+        
+        Args:
+            frames (list): List of frames to analyze
+            prompt (str): Prompt describing what anomaly to look for
+            callback (function, optional): Callback function to report progress
+            
+        Returns:
+            dict: Cumulative analysis result
+        """
+        # First, analyze each frame individually to identify potential anomalies
+        individual_results = []
+        for i, frame in enumerate(frames):
+            if callback:
+                callback(i, len(frames) * 2)  # First half of progress for individual analysis
+                
+            result = self.analyze_frame(frame, f"{prompt} Provide a brief analysis of this frame only.")
+            individual_results.append(result)
+        
+        # Identify frames with potential anomalies
+        anomaly_frames = []
+        anomaly_descriptions = []
+        anomaly_types = []
+        
+        for i, result in enumerate(individual_results):
+            if "error" not in result and result["anomaly_detected"]:
+                anomaly_frames.append(result["frame"])
+                anomaly_descriptions.append(f"Frame {i+1}: {result['text']}")
+                anomaly_types.append(result["anomaly_type"])
+                
+                # Limit to 3 anomaly frames
+                if len(anomaly_frames) >= 3:
+                    break
+        
+        # If no anomalies were detected, use the first, middle, and last frames
+        if not anomaly_frames and len(frames) > 0:
+            if len(frames) == 1:
+                anomaly_frames = [frames[0]]
+            elif len(frames) == 2:
+                anomaly_frames = [frames[0], frames[1]]
+            else:
+                anomaly_frames = [
+                    frames[0],
+                    frames[len(frames) // 2],
+                    frames[-1]
+                ]
+        
+        # Limit to max 3 frames
+        anomaly_frames = anomaly_frames[:3]
+        
+        # Create a cumulative analysis prompt with the anomaly descriptions
+        cumulative_prompt = f"""
+{prompt}
+
+Based on the analysis of all frames, provide a comprehensive summary of any anomalies detected in the video. Focus on patterns or recurring issues. Here are some notable observations from individual frames:
+
+{chr(10).join(anomaly_descriptions[:5])}
+
+After your analysis, please include a structured assessment at the end of your response in this exact format:
+ANOMALY_DETECTED: [Yes/No]
+ANOMALY_TYPE: [Human/Non-human/None]
+
+For ANOMALY_DETECTED, answer "Yes" if you detect any anomaly across the video, otherwise "No".
+For ANOMALY_TYPE, if an anomaly is detected, classify the predominant type as either "Human" (if it involves people or human activities) or "Non-human" (if it involves objects, animals, or environmental factors). If no anomaly is detected, use "None".
+"""
+        
+        try:
+            if callback:
+                callback(len(frames), len(frames) * 2)  # Second half of progress for cumulative analysis
+            
+            # Encode the selected frames
+            base64_images = [self.encode_image_to_base64(frame) for frame in anomaly_frames]
+            
+            # Create the content for the API call
+            content = [{"type": "text", "text": cumulative_prompt}]
+            
+            # Add the images
+            for base64_image in base64_images:
+                content.append({
+                    "type": "image_url",
+                    "image_url": {
+                        "url": f"data:image/png;base64,{base64_image}"
+                    }
+                })
+            
+            response = self.client.chat.completions.create(
+                model=self.model,
+                messages=[
+                    {
+                        "role": "user",
+                        "content": content
+                    }
+                ],
+                max_tokens=1500
+            )
+            
+            # Extract the response text
+            response_text = response.choices[0].message.content
+            
+            # Extract anomaly detection information using regex
+            anomaly_detected = False
+            anomaly_type = "None"
+            
+            # Look for the structured format
+            anomaly_match = re.search(r'ANOMALY_DETECTED:\s*(Yes|No)', response_text, re.IGNORECASE)
+            if anomaly_match and anomaly_match.group(1).lower() == 'yes':
+                anomaly_detected = True
+                
+                # If anomaly detected, look for the type
+                type_match = re.search(r'ANOMALY_TYPE:\s*(Human|Non-human|None)', response_text, re.IGNORECASE)
+                if type_match:
+                    anomaly_type = type_match.group(1)
+            
+            return {
+                "text": response_text,
+                "frames": anomaly_frames,
+                "anomaly_detected": anomaly_detected,
+                "anomaly_type": anomaly_type,
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+        except Exception as e:
+            return {
+                "error": str(e),
+                "frames": anomaly_frames,
+                "anomaly_detected": False,
+                "anomaly_type": "None",
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+    
+    def process_video(self, video_path, skip_frames, prompt, analysis_depth="granular", callback=None):
+        """
+        Process a video file, extracting frames and analyzing them for anomalies.
+        
+        Args:
+            video_path (str): Path to the video file
+            skip_frames (int): Number of frames to skip between captures
+            prompt (str): Prompt describing what anomaly to look for
+            analysis_depth (str): "granular" for frame-by-frame analysis or "cumulative" for overall analysis
+            callback (function, optional): Callback function to report progress
+            
+        Returns:
+            list or dict: List of analysis results for each processed frame (granular) or dict with cumulative analysis (cumulative)
+        """
+        frames = self.extract_frames(video_path, skip_frames)
+        
+        if analysis_depth == "cumulative":
+            return self.analyze_frames_cumulatively(frames, prompt, callback)
+        else:  # granular (default)
+            results = []
+            
+            for i, frame in enumerate(frames):
+                if callback:
+                    if analysis_depth == "cumulative":
+                        callback(i, len(frames) / 2)
+                    else:
+                        callback(i, len(frames))
+                    
+                result = self.analyze_frame(frame, prompt)
+                results.append(result)
+                
+            return results

phi4_detector.py

@@ -0,0 +1,500 @@
+"""
+Phi-4 model implementation for video anomaly detection using Hugging Face transformers.
+"""
+
+import cv2
+import numpy as np
+import base64
+import os
+import tempfile
+from PIL import Image
+import io
+import re
+import torch
+import time
+from transformers import AutoModelForCausalLM, AutoTokenizer, AutoProcessor
+from huggingface_hub import snapshot_download
+
+class Phi4AnomalyDetector:
+    def __init__(self, model_name="microsoft/Phi-4-multimodal-instruct"):
+        """
+        Initialize the Phi4AnomalyDetector with the Phi-4 vision model.
+        
+        Args:
+            model_name (str): Name of the Phi-4 vision model on Hugging Face
+        """
+        self.model_name = model_name
+        self.model_dir = os.path.join(os.getcwd(), "phi4_model")
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        
+        # Load or download the model
+        self.load_model()
+        
+    def load_model(self):
+        """
+        Load the Phi-4 model from local directory or download from Hugging Face.
+        """
+        try:
+            if not os.path.exists(self.model_dir):
+                print(f"Downloading {self.model_name} model to {self.model_dir}...")
+                snapshot_download(repo_id=self.model_name, local_dir=self.model_dir)
+                print("Model downloaded successfully.")
+            else:
+                print(f"Using existing model from {self.model_dir}")
+            
+            # Load model components with trust_remote_code=True
+            self.processor = AutoProcessor.from_pretrained(
+                self.model_dir,
+                trust_remote_code=True
+            )
+            
+            self.tokenizer = AutoTokenizer.from_pretrained(
+                self.model_dir,
+                trust_remote_code=True
+            )
+            
+            # Load model with appropriate dtype based on device
+            if self.device == "cuda":
+                self.model = AutoModelForCausalLM.from_pretrained(
+                    self.model_dir,
+                    torch_dtype=torch.float16,
+                    device_map="auto",
+                    trust_remote_code=True
+                )
+            else:
+                self.model = AutoModelForCausalLM.from_pretrained(
+                    self.model_dir,
+                    device_map="auto",
+                    trust_remote_code=True
+                )
+            
+            print(f"Phi-4 model loaded successfully on {self.device}")
+            
+        except Exception as e:
+            raise RuntimeError(f"Failed to load Phi-4 model: {str(e)}")
+    
+    def extract_frames(self, video_path, skip_frames):
+        """
+        Extract frames from a video file, skipping the specified number of frames.
+        
+        Args:
+            video_path (str): Path to the video file
+            skip_frames (int): Number of frames to skip between captures
+            
+        Returns:
+            list: List of extracted frames as numpy arrays
+        """
+        frames = []
+        # Use the default backend for video files
+        # DirectShow can cause issues with some video files
+        cap = cv2.VideoCapture(video_path)
+        
+        # Don't set MJPG format for video files as it can interfere with proper decoding
+        # cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M','J','P','G'))
+        
+        if not cap.isOpened():
+            raise ValueError(f"Could not open video file: {video_path}")
+        
+        frame_count = 0
+        while True:
+            ret, frame = cap.read()
+            if not ret:
+                break
+                
+            if frame_count % (skip_frames + 1) == 0:
+                # Convert from BGR to RGB
+                rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                frames.append(rgb_frame)
+                
+            frame_count += 1
+            
+        cap.release()
+        return frames
+    
+    def process_live_stream(self, stream_source, skip_frames, prompt, analysis_depth="granular", max_frames=None, callback=None, time_interval=None):
+        """
+        Process frames from a live video stream.
+        
+        Args:
+            stream_source: Stream source (0 for webcam, URL for IP camera or RTSP stream)
+            skip_frames (int): Number of frames to skip between captures
+            prompt (str): Prompt describing what anomaly to look for
+            analysis_depth (str): "granular" for frame-by-frame analysis or "cumulative" for overall analysis
+            max_frames (int, optional): Maximum number of frames to process (None for unlimited)
+            callback (function, optional): Callback function to report progress
+            time_interval (int, optional): If set, capture one frame every X seconds instead of using skip_frames
+            
+        Returns:
+            list or dict: List of analysis results for each processed frame (granular) or dict with cumulative analysis (cumulative)
+        """
+        # Open the video stream with appropriate backend
+        # Only use DirectShow for local webcams (0 or 1) on Windows
+        if os.name == 'nt' and (stream_source == 0 or stream_source == 1):
+            # This is a local webcam on Windows, use DirectShow
+            cap = cv2.VideoCapture(stream_source, cv2.CAP_DSHOW)
+            
+            # For webcams, MJPG format can be more stable
+            cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M','J','P','G'))
+        else:
+            # For IP cameras, RTSP streams, or non-Windows systems, use default backend
+            cap = cv2.VideoCapture(stream_source)
+        
+        if not cap.isOpened():
+            raise ValueError(f"Could not open video stream: {stream_source}")
+        
+        frames = []
+        frame_count = 0
+        processed_count = 0
+        last_capture_time = time.time()
+        
+        try:
+            while True:
+                ret, frame = cap.read()
+                if not ret:
+                    break
+                
+                current_time = time.time()
+                
+                # Determine if we should capture this frame
+                should_capture = False
+                
+                if time_interval is not None:
+                    # Time-based interval mode
+                    if current_time - last_capture_time >= time_interval:
+                        should_capture = True
+                        last_capture_time = current_time
+                else:
+                    # Frame-skip mode
+                    if frame_count % (skip_frames + 1) == 0:
+                        should_capture = True
+                
+                if should_capture:
+                    # Convert from BGR to RGB
+                    rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                    frames.append(rgb_frame)
+                    processed_count += 1
+                    
+                    # Process the frame immediately in time interval mode
+                    if time_interval is not None:
+                        # Process the frame immediately
+                        result = self.analyze_frame(rgb_frame, prompt)
+                        
+                        # Make sure the frame is included in the result
+                        if "frame" not in result:
+                            result["frame"] = rgb_frame
+                        
+                        # If we have a callback, call it with -1 for total to indicate continuous mode
+                        if callback:
+                            callback(processed_count, -1)
+                        
+                        # In time interval mode, we yield results one by one
+                        if analysis_depth == "granular":
+                            yield result
+                        else:
+                            # For cumulative analysis, we need to keep all frames
+                            # We'll handle this outside the loop
+                            pass
+                    else:
+                        # Update progress if callback is provided (frame count mode)
+                        if callback and max_frames:
+                            callback(processed_count, max_frames)
+                        
+                        # Break if we've reached the maximum number of frames
+                        if max_frames and processed_count >= max_frames:
+                            break
+                
+                frame_count += 1
+        finally:
+            cap.release()
+        
+        # If we're in time interval mode with cumulative analysis, we don't return here
+        # as we're yielding results above
+        if time_interval is not None and analysis_depth == "cumulative":
+            # This is a special case - we need to periodically do cumulative analysis
+            # For simplicity, we'll just return the current cumulative analysis
+            result = self.analyze_frames_cumulatively(frames, prompt, callback)
+            yield result
+            return
+            
+        # Process the collected frames for non-time interval mode
+        if time_interval is None:
+            if analysis_depth == "cumulative":
+                return self.analyze_frames_cumulatively(frames, prompt, callback)
+            else:  # granular (default)
+                results = []
+                
+                for i, frame in enumerate(frames):
+                    if callback:
+                        callback(i, len(frames))
+                        
+                    result = self.analyze_frame(frame, prompt)
+                    results.append(result)
+                    
+                return results
+    
+    def analyze_frame(self, frame, prompt):
+        """
+        Analyze a frame using the Phi-4 vision model.
+        
+        Args:
+            frame (numpy.ndarray): Frame to analyze
+            prompt (str): Prompt describing what anomaly to look for
+            
+        Returns:
+            dict: Analysis result from the model
+        """
+        # Convert numpy array to PIL Image
+        pil_image = Image.fromarray(frame)
+        
+        # Enhanced prompt to get structured information about anomalies
+        enhanced_prompt = f"""
+{prompt}
+
+After your analysis, please include a structured assessment at the end of your response in this exact format:
+ANOMALY_DETECTED: [Yes/No]
+ANOMALY_TYPE: [Human/Non-human/None]
+CONFIDENCE: [0-100]
+
+For ANOMALY_DETECTED, answer "Yes" if you detect any anomaly, otherwise "No".
+For ANOMALY_TYPE, if an anomaly is detected, classify it as either "Human" (if it involves people or human activities) or "Non-human" (if it involves objects, animals, or environmental factors). If no anomaly is detected, use "None".
+For CONFIDENCE, provide a number from 0 to 100 indicating your confidence level in the assessment.
+"""
+        
+        try:
+            # Process the image and prompt with the Phi-4 model
+            inputs = self.processor(text=enhanced_prompt, images=pil_image, return_tensors="pt")
+            
+            # Move inputs to the same device as the model
+            for key in inputs:
+                if torch.is_tensor(inputs[key]):
+                    inputs[key] = inputs[key].to(self.model.device)
+            
+            # Generate response
+            with torch.no_grad():
+                outputs = self.model.generate(
+                    **inputs,
+                    max_new_tokens=500,
+                    do_sample=False
+                )
+            
+            # Decode the response
+            response_text = self.tokenizer.decode(outputs[0], skip_special_tokens=True)
+            
+            # Extract the part after the prompt
+            if enhanced_prompt in response_text:
+                response_text = response_text.split(enhanced_prompt)[-1].strip()
+            
+            # Extract anomaly detection information using regex
+            anomaly_detected = False
+            anomaly_type = "None"
+            confidence = 0
+            
+            # Look for the structured format
+            anomaly_match = re.search(r'ANOMALY_DETECTED:\s*(Yes|No)', response_text, re.IGNORECASE)
+            if anomaly_match and anomaly_match.group(1).lower() == 'yes':
+                anomaly_detected = True
+                confidence = 90  # Default high confidence when anomaly is detected
+                
+                # If anomaly detected, look for the type
+                type_match = re.search(r'ANOMALY_TYPE:\s*(Human|Non-human|None)', response_text, re.IGNORECASE)
+                if type_match:
+                    anomaly_type = type_match.group(1)
+                    
+                # Look for confidence information
+                conf_match = re.search(r'CONFIDENCE:\s*(\d+)', response_text, re.IGNORECASE)
+                if conf_match:
+                    try:
+                        confidence = int(conf_match.group(1))
+                    except:
+                        pass  # Keep default confidence if parsing fails
+            
+            return {
+                "text": response_text,
+                "analysis": response_text,  # Add analysis field as an alias for text
+                "frame": frame,
+                "anomaly_detected": anomaly_detected,
+                "anomaly_type": anomaly_type,
+                "confidence": confidence,  # Add confidence field
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+        except Exception as e:
+            return {
+                "error": str(e),
+                "frame": frame,
+                "anomaly_detected": False,
+                "anomaly_type": "None",
+                "confidence": 0,  # Add default confidence for error
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+    
+    def analyze_frames_cumulatively(self, frames, prompt, callback=None):
+        """
+        Analyze all frames together and provide a cumulative analysis.
+        
+        Args:
+            frames (list): List of frames to analyze
+            prompt (str): Prompt describing what anomaly to look for
+            callback (function, optional): Callback function to report progress
+            
+        Returns:
+            dict: Cumulative analysis result
+        """
+        # First, analyze each frame individually to identify potential anomalies
+        individual_results = []
+        for i, frame in enumerate(frames):
+            if callback:
+                callback(i, len(frames) * 2)  # First half of progress for individual analysis
+                
+            result = self.analyze_frame(frame, f"{prompt} Provide a brief analysis of this frame only.")
+            individual_results.append(result)
+        
+        # Identify frames with potential anomalies
+        anomaly_frames = []
+        anomaly_descriptions = []
+        anomaly_types = []
+        
+        for i, result in enumerate(individual_results):
+            if "error" not in result and result["anomaly_detected"]:
+                anomaly_frames.append(result["frame"])
+                anomaly_descriptions.append(f"Frame {i+1}: {result['text']}")
+                anomaly_types.append(result["anomaly_type"])
+                
+                # Limit to 3 anomaly frames
+                if len(anomaly_frames) >= 3:
+                    break
+        
+        # If no anomalies were detected, use the first, middle, and last frames
+        if not anomaly_frames and len(frames) > 0:
+            if len(frames) == 1:
+                anomaly_frames = [frames[0]]
+            elif len(frames) == 2:
+                anomaly_frames = [frames[0], frames[1]]
+            else:
+                anomaly_frames = [
+                    frames[0],
+                    frames[len(frames) // 2],
+                    frames[-1]
+                ]
+        
+        # Limit to max 3 frames
+        anomaly_frames = anomaly_frames[:3]
+        
+        # Create a cumulative analysis prompt with the anomaly descriptions
+        cumulative_prompt = f"""
+{prompt}
+
+Based on the analysis of all frames, provide a comprehensive summary of any anomalies detected in the video. Focus on patterns or recurring issues. Here are some notable observations from individual frames:
+
+{chr(10).join(anomaly_descriptions[:5])}
+
+After your analysis, please include a structured assessment at the end of your response in this exact format:
+ANOMALY_DETECTED: [Yes/No]
+ANOMALY_TYPE: [Human/Non-human/None]
+
+For ANOMALY_DETECTED, answer "Yes" if you detect any anomaly across the video, otherwise "No".
+For ANOMALY_TYPE, if an anomaly is detected, classify the predominant type as either "Human" (if it involves people or human activities) or "Non-human" (if it involves objects, animals, or environmental factors). If no anomaly is detected, use "None".
+"""
+        
+        try:
+            if callback:
+                callback(len(frames), len(frames) * 2)  # Second half of progress for cumulative analysis
+            
+            # For cumulative analysis, we'll use the first anomaly frame (or first frame if no anomalies)
+            representative_frame = anomaly_frames[0] if anomaly_frames else frames[0]
+            pil_image = Image.fromarray(representative_frame)
+            
+            # Process with Phi-4
+            inputs = self.processor(text=cumulative_prompt, images=pil_image, return_tensors="pt")
+            
+            # Move inputs to the same device as the model
+            for key in inputs:
+                if torch.is_tensor(inputs[key]):
+                    inputs[key] = inputs[key].to(self.model.device)
+            
+            # Generate response
+            with torch.no_grad():
+                outputs = self.model.generate(
+                    **inputs,
+                    max_new_tokens=1000,
+                    do_sample=False
+                )
+            
+            # Decode the response
+            response_text = self.tokenizer.decode(outputs[0], skip_special_tokens=True)
+            
+            # Extract the part after the prompt
+            if cumulative_prompt in response_text:
+                response_text = response_text.split(cumulative_prompt)[-1].strip()
+            
+            # Extract anomaly detection information using regex
+            anomaly_detected = False
+            anomaly_type = "None"
+            confidence = 0
+            
+            # Look for the structured format
+            anomaly_match = re.search(r'ANOMALY_DETECTED:\s*(Yes|No)', response_text, re.IGNORECASE)
+            if anomaly_match and anomaly_match.group(1).lower() == 'yes':
+                anomaly_detected = True
+                confidence = 90  # Default high confidence when anomaly is detected
+                
+                # If anomaly detected, look for the type
+                type_match = re.search(r'ANOMALY_TYPE:\s*(Human|Non-human|None)', response_text, re.IGNORECASE)
+                if type_match:
+                    anomaly_type = type_match.group(1)
+                    
+                # Look for confidence information
+                conf_match = re.search(r'CONFIDENCE:\s*(\d+)', response_text, re.IGNORECASE)
+                if conf_match:
+                    try:
+                        confidence = int(conf_match.group(1))
+                    except:
+                        pass  # Keep default confidence if parsing fails
+            
+            return {
+                "text": response_text,
+                "analysis": response_text,  # Add analysis field as an alias for text
+                "frames": anomaly_frames,
+                "anomaly_detected": anomaly_detected,
+                "anomaly_type": anomaly_type,
+                "confidence": confidence,  # Add confidence field
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+        except Exception as e:
+            return {
+                "error": str(e),
+                "frames": anomaly_frames,
+                "anomaly_detected": False,
+                "anomaly_type": "None",
+                "confidence": 0,  # Add default confidence for error
+                "timestamp": time.time()  # Add timestamp for live stream analysis
+            }
+    
+    def process_video(self, video_path, skip_frames, prompt, analysis_depth="granular", callback=None):
+        """
+        Process a video file, extracting frames and analyzing them for anomalies.
+        
+        Args:
+            video_path (str): Path to the video file
+            skip_frames (int): Number of frames to skip between captures
+            prompt (str): Prompt describing what anomaly to look for
+            analysis_depth (str): "granular" for frame-by-frame analysis or "cumulative" for overall analysis
+            callback (function, optional): Callback function to report progress
+            
+        Returns:
+            list or dict: List of analysis results for each processed frame (granular) or dict with cumulative analysis (cumulative)
+        """
+        frames = self.extract_frames(video_path, skip_frames)
+        
+        if analysis_depth == "cumulative":
+            return self.analyze_frames_cumulatively(frames, prompt, callback)
+        else:  # granular (default)
+            results = []
+            
+            for i, frame in enumerate(frames):
+                if callback:
+                    callback(i, len(frames))
+                    
+                result = self.analyze_frame(frame, prompt)
+                results.append(result)
+                
+            return results 

requirements.txt

@@ -0,0 +1,12 @@
+opencv-python==4.8.1.78
+numpy==1.26.2
+streamlit==1.31.0
+python-dotenv==1.0.0
+openai==1.3.7
+Pillow==10.1.0
+requests==2.31.0
+colorama==0.4.6
+torch>=2.0.0
+transformers>=4.35.0
+huggingface-hub>=0.19.0
+accelerate>=0.25.0 

vid-1.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7f36ae934376c55a7ce02f06a48db1d1d1acd8e7a2ac254c131051aaacb7339
+size 2725255

vid-2.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e7f944017e7dc1d9c0f5b38338996e80f26d55535fffbfd232c6e16d2386d8a
+size 880606