π Lesson 5
D3
Cloud Motion Vector Estimation from Hemispherical Imagery
It's a way to track how clouds move across the sky by analyzing sequences of fish-eye camera images taken from the ground.
π― Learning Objectives
- β Calculate cloud motion vectors using Lucas-Kanade optical flow on registered hemispherical image sequences
- β Design a preprocessing pipeline to correct lens distortion and remove obstructions (e.g., poles, birds) from all-sky imagery
- β Analyze CMV accuracy by comparing estimated vectors against ground-truth satellite-derived cloud drift velocities
- β Apply spatial-temporal filtering to suppress noise and false vectors caused by thin cirrus or camera artifacts
- β Explain the impact of CMV uncertainty on 1β15 minute photovoltaic power forecast skill scores
π Why This Matters
In AI-augmented renewable forecasting, predicting solar irradiance at sub-hour scales hinges on anticipating cloud cover changes β the dominant source of PV generation volatility. Unlike satellites (with ~5β15 min latency and coarse resolution), ground-based hemispherical imagers provide <30-second temporal resolution and sub-kilometer spatial fidelity. Estimating *where* and *how fast* clouds move β i.e., computing motion vectors β enables physics-informed nowcasting that directly improves grid stability, reduces reserve requirements, and increases renewable dispatch confidence.
π Core Principles
CMV estimation begins with geometric correction: raw fisheye images must be undistorted and mapped to a Lambert azimuthal equal-area projection to preserve angular relationships. Next, temporal image alignment compensates for minor camera shifts or thermal drift. Motion estimation then proceeds via either feature-based methods (e.g., SIFT/ORB keypoints matched across frames) or dense optical flow (e.g., Lucas-Kanade or FarnebΓ€ck). Vectors are expressed in azimuthβelevation space or projected onto a local tangent plane (u,v) in meters/second. Finally, physical consistency filters (e.g., median-of-neighbors, divergence constraints, maximum speed thresholds) remove outliers. The resulting field is interpolated and assimilated into irradiance forecast models via advection kernels or deep learning encoders.
π Lucas-Kanade Optical Flow (Local)
Lucas-Kanade computes pixel-level displacement (u,v) by solving the brightness constancy equation under a local smoothness assumption. Itβs robust for small displacements (<10 pixels/frame) typical in high-frame-rate (1β2 Hz) all-sky video.
π‘ Worked Example
Problem: Given two consecutive 1024Γ1024 hemispherical images (Iβ, Iβ) captured at 1 Hz; at pixel (x,y) = (512,512), βI = [β12.3, 8.7] (intensity gradient), Iβ = β41.6 (temporal derivative), and the 2Γ2 structure tensor M = [[154.2, β92.1], [β92.1, 138.5]]. Compute the optical flow vector (u,v).
1.
Step 1: Confirm invertibility of M: det(M) = (154.2)(138.5) β (β92.1)Β² = 21357 β 8482 = 12875 > 0 β invertible.
2.
Step 2: Compute Mβ»ΒΉ = (1/det(M)) Γ [[138.5, 92.1], [92.1, 154.2]] β [[0.01075, 0.00715], [0.00715, 0.01200]].
3.
Step 3: Compute [u;v] = βMβ»ΒΉ βI Iβ = βMβ»ΒΉ Γ [β12.3; 8.7] Γ (β41.6) β first compute βI Iβ = [511.68; β361.92], then apply Mβ»ΒΉ: u β 0.01075Γ511.68 + 0.00715Γ(β361.92) β 5.49 β 2.59 = 2.90; v β 0.00715Γ511.68 + 0.01200Γ(β361.92) β 3.66 β 4.34 = β0.68.
4.
Step 4: Convert pixel displacement to physical velocity: assuming 0.1Β°/pixel resolution and 1 s frame interval, u = 2.90 px/s Γ 0.1Β°/px Γ (Ο/180) rad/Β° Γ Rβarth Γ cos(lat) β 1.4 m/s (azimuth); v β β0.3 m/s (elevation).
Answer:
The estimated local motion vector is (u,v) β (2.90, β0.68) px/s, corresponding to ~1.4 m/s eastward and ~0.3 m/s downward in local tangent coordinates β physically plausible for low-stratus advection at mid-latitudes.
ποΈ Real-World Application
At the National Renewable Energy Laboratoryβs (NREL) Solar Radiation Research Laboratory (SRRL) in Golden, CO, a network of 8 SkyCam systems (using Canon EOS R6 fisheye + 8mm lens) feeds CMV estimates into the ADVECT nowcasting model. During the 2022 Western Wind and Solar Integration Study, CMV-augmented forecasts reduced 5-min PV power RMSE by 22% compared to persistence alone. Vectors were computed at 2 Hz, filtered using a 5Γ5 median + divergence <0.005 sβ»ΒΉ constraint, and fused with GOES-18 ABI cloud-top height data to resolve vertical shear ambiguity β enabling accurate ramp detection for utility-scale plants within 5 km radius.
π§ Interactive Calculator
π§ Open AI-Augmented Renewable Forecasting Infrastructure Calculatorπ Case Connection
π CAISO Zone 12 Solar Ramp Event Mitigation
Unpredicted cloud-edge ramp rates exceeding 150 MW/min causing reserve shortfalls
π Hawaii Island Microgrid Solar Forecast Hardening
Volcanic haze and trade-wind cloud variability caused 30β50% forecast error spikes during monsoon season