SmartCane/r_app/experiments/10_CI_report_with_kpis.Rmd

---
params:
  ref: "word-styles-reference-var1.docx"
  output_file: CI_report_with_kpis.docx
  report_date: "2025-09-18"
  data_dir: "esa"
  mail_day: "Wednesday"
  borders: FALSE
  ci_plot_type: "both"  # options: "absolute", "cumulative", "both"
  colorblind_friendly: TRUE  # use colorblind-friendly palettes (viridis/plasma)
  facet_by_season: FALSE  # facet CI trend plots by season instead of overlaying
  x_axis_unit: "days"  # x-axis unit for trend plots: "days" or "weeks"
output:
 # html_document:
 #   toc: yes
 #   df_print: paged  
   word_document:
    reference_docx: !expr file.path("word-styles-reference-var1.docx")
    toc: no
editor_options:
  chunk_output_type: console
---

```{r setup_parameters, include=FALSE}
# Set up basic report parameters from input values
report_date <- params$report_date
mail_day <- params$mail_day
borders <- params$borders
ci_plot_type <- params$ci_plot_type
colorblind_friendly <- params$colorblind_friendly
facet_by_season <- params$facet_by_season
x_axis_unit <- params$x_axis_unit
```

```{r load_libraries, message=FALSE, warning=FALSE, include=FALSE}
# Configure knitr options
knitr::opts_chunk$set(warning = FALSE, message = FALSE)

# Load all packages at once with suppressPackageStartupMessages
suppressPackageStartupMessages({
  library(here)
  library(sf)
  library(terra)
  library(exactextractr)
  library(tidyverse)
  library(tmap)
  library(lubridate)
  library(zoo)
  library(rsample)
  library(caret)
  library(randomForest)
  library(CAST)
  library(knitr)
})

# Load custom utility functions
tryCatch({
  source("report_utils.R")
}, error = function(e) {
  message(paste("Error loading report_utils.R:", e$message))
  # Try alternative path if the first one fails
  tryCatch({
    source(here::here("r_app", "report_utils.R"))
  }, error = function(e) {
    stop("Could not load report_utils.R from either location: ", e$message)
  })
})
```

```{r initialize_project_config, message=FALSE, warning=FALSE, include=FALSE}
# Set the project directory from parameters
project_dir <- params$data_dir

# Source project parameters with error handling
tryCatch({
  source(here::here("r_app", "parameters_project.R"))
}, error = function(e) {
  stop("Error loading parameters_project.R: ", e$message)
})

# Log initial configuration
safe_log("Starting the R Markdown script with KPIs")
safe_log(paste("mail_day params:", params$mail_day))
safe_log(paste("report_date params:", params$report_date))
safe_log(paste("mail_day variable:", mail_day))
```

```{r load_kpi_data, message=FALSE, warning=FALSE, include=FALSE}
# SIMPLE KPI LOADING - just load the damn files!
kpi_data_dir <- file.path("..", "laravel_app", "storage", "app", project_dir, "reports", "kpis")
date_suffix <- format(as.Date(report_date), "%Y%m%d")
summary_file <- file.path(kpi_data_dir, paste0(project_dir, "_kpi_summary_tables_", date_suffix, ".rds"))

# Load the summary tables (this works!)
summary_tables <- readRDS(summary_file)

# Load field details too
field_details_file <- file.path(kpi_data_dir, paste0(project_dir, "_field_details_", date_suffix, ".rds"))
field_details_table <- readRDS(field_details_file)

# Set this for compatibility with rest of report
kpi_files_exist <- TRUE

safe_log("✓ KPI summary tables loaded successfully")

```

```{r calculate_dates_and_weeks, message=FALSE, warning=FALSE, include=FALSE}
# Set locale for consistent date formatting
Sys.setlocale("LC_TIME", "C")

# Initialize date variables from parameters
today <- as.character(report_date)
mail_day_as_character <- as.character(mail_day)

# Calculate report dates and weeks
report_date_obj <- as.Date(today)
current_week <- as.numeric(format(report_date_obj, "%U"))
year <- as.numeric(format(report_date_obj, "%Y"))

# Calculate dates for weekly analysis
week_start <- report_date_obj - ((as.numeric(format(report_date_obj, "%w")) + 1) %% 7)
week_end <- week_start + 6

safe_log(paste("Report week:", current_week, "Year:", year))
safe_log(paste("Week range:", week_start, "to", week_end))
```

# SmartCane Monitoring Report with KPIs

**Report Date:** `r format(as.Date(report_date), "%B %d, %Y")`  
**Project:** `r toupper(project_dir)`  
**Week:** `r current_week` of `r year`

---

## Executive Summary - Key Performance Indicators

This report provides a comprehensive analysis of sugarcane field performance using satellite-based monitoring.

### Field Uniformity
```{r field_uniformity_table, echo=FALSE}
kable(summary_tables$field_uniformity_summary, 
      caption = "Field Uniformity Summary",
      col.names = c("Uniformity Level", "Count", "Percent"))
```

### TCH Forecasted
```{r tch_forecasted_table, echo=FALSE}
kable(summary_tables$tch_forecasted_summary,
      caption = "TCH Forecasted Summary",
      col.names = c("Field Groups", "Count", "Value"))
```

### Farm-wide Area Change
```{r area_change_table, echo=FALSE}
kable(summary_tables$area_change_summary,
      caption = "Farm-wide Area Change Summary", 
      col.names = c("Change Type", "Hectares", "Percent"))
```

### Weed Presence Score
```{r weed_presence_table, echo=FALSE}
kable(summary_tables$weed_presence_summary,
      caption = "Weed Presence Score Summary",
      col.names = c("Weed Risk Level", "Field Count", "Percent"))
```

### Growth Decline Index
```{r growth_decline_table, echo=FALSE}
kable(summary_tables$growth_decline_summary,
      caption = "Growth Decline Index Summary",
      col.names = c("Risk Level", "Count", "Percent"))
```

### Gap Filling Assessment
```{r gap_filling_table, echo=FALSE}
kable(summary_tables$gap_filling_summary,
      caption = "Gap Filling Assessment Summary",
      col.names = c("Gap Level", "Field Count", "Percent"))
```

### Detailed KPI Breakdown

```{r kpi_detailed_breakdown, echo=FALSE}
# Show all 6 KPI tables in a more compact format
cat("**Field Uniformity**\n")
kable(summary_tables$field_uniformity_summary, col.names = c("Level", "Count", "%"))

cat("\n**TCH Forecasted**\n")  
kable(summary_tables$tch_forecasted_summary, col.names = c("Groups", "Count", "Value"))

cat("\n**Area Change**\n")
kable(summary_tables$area_change_summary, col.names = c("Change", "Ha", "%"))

cat("\n**Weed Presence**\n")
kable(summary_tables$weed_presence_summary, col.names = c("Risk", "Count", "%"))

cat("\n**Growth Decline**\n")
kable(summary_tables$growth_decline_summary, col.names = c("Risk", "Count", "%"))

cat("\n**Gap Filling**\n")
kable(summary_tables$gap_filling_summary, col.names = c("Level", "Count", "%"))
```

## KPI Summary Charts

```{r kpi_charts, echo=FALSE, fig.width=10, fig.height=8}
# Load ggplot2 for creating charts
library(ggplot2)
library(gridExtra)

# Create charts for key KPIs using correct column names
# 1. Field Uniformity Chart
p1 <- ggplot(summary_tables$field_uniformity_summary, aes(x = reorder(`Uniformity Level`, -Count), y = Count)) +
  geom_col(fill = "steelblue", alpha = 0.7) +
  labs(title = "Field Uniformity Distribution", x = "Uniformity Level", y = "Field Count") +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

# 2. TCH Forecasted Chart  
p2 <- ggplot(summary_tables$tch_forecasted_summary, aes(x = `Field Groups`, y = Value)) +
  geom_col(fill = "darkgreen", alpha = 0.7) +
  labs(title = "TCH Forecast by Field Groups", x = "Field Groups", y = "Value") +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

# 3. Growth Decline Risk Chart
p3 <- ggplot(summary_tables$growth_decline_summary, aes(x = reorder(`Risk Level`, -Count), y = Count)) +
  geom_col(fill = "orange", alpha = 0.7) +
  labs(title = "Growth Decline Risk Distribution", x = "Risk Level", y = "Field Count") +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

# 4. Weed Presence Risk Chart
p4 <- ggplot(summary_tables$weed_presence_summary, aes(x = reorder(`Weed Risk Level`, -`Field Count`), y = `Field Count`)) +
  geom_col(fill = "red", alpha = 0.7) +
  labs(title = "Weed Presence Risk Distribution", x = "Risk Level", y = "Field Count") +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

# Arrange plots in a grid
grid.arrange(p1, p2, p3, p4, ncol = 2, nrow = 2)
```

---

\newpage

## Field-by-Field Analysis

The following sections provide detailed analysis for each monitored field, including spatial maps, temporal trends, and field-specific KPI summaries.

```{r load_field_data, message=FALSE, warning=FALSE, include=FALSE}
# Load field data and prepare for field-by-field analysis
# Load the spatial and temporal CI data needed for visualizations

# Check if the required data objects exist from parameters_project.R
required_objects <- c("AllPivots0", "CI", "CI_m1", "CI_m2", "CI_quadrant", "harvesting_data")
missing_objects <- required_objects[!sapply(required_objects, exists)]

if (length(missing_objects) > 0) {
  safe_log(paste("Missing required objects for field analysis:", paste(missing_objects, collapse = ", ")), "WARNING")
  field_analysis_possible <- FALSE
} else {
  safe_log("All required data objects found for field analysis")
  field_analysis_possible <- TRUE
  
  # Prepare field list from the loaded boundaries
  field_list <- AllPivots0 %>%
    filter(!is.na(field), !is.na(sub_field)) %>%
    group_by(field) %>%
    summarise(.groups = 'drop') %>%
    slice_head(n = 3)  # Limit to first 3 fields for report length
}
```

```{r generate_field_visualizations, eval=TRUE, fig.height=3.8, fig.width=10, message=FALSE, echo=FALSE, warning=FALSE, include=TRUE, results='asis'}
# Generate detailed visualizations for each field (copied from 05_CI_report_dashboard_planet.Rmd)
if (field_analysis_possible) {
  tryCatch({
    # Merge field polygons for processing and filter out NA field names
    AllPivots_merged <- AllPivots0 %>% 
      dplyr::filter(!is.na(field), !is.na(sub_field)) %>%  # Filter out NA fields
      dplyr::group_by(field) %>% 
      dplyr::summarise(.groups = 'drop') %>%
      slice_head(n = 3)  # Limit to first 3 fields for report
    
    # Generate plots for each field
    for(i in seq_along(AllPivots_merged$field)) {
      field_name <- AllPivots_merged$field[i]
      
      # Skip if field_name is still NA (double check)
      if(is.na(field_name)) {
        next
      }
      
      tryCatch({
        # Add page break before each field (except the first one)
        if(i > 1) {
          cat("\\newpage\n\n")
        }
        
        # Call ci_plot with explicit parameters (ci_plot will generate its own header)
        ci_plot(
          pivotName = field_name,
          field_boundaries = AllPivots0,
          current_ci = CI,
          ci_minus_1 = CI_m1,
          ci_minus_2 = CI_m2,
          last_week_diff = last_week_dif_raster_abs,
          three_week_diff = three_week_dif_raster_abs,
          harvesting_data = harvesting_data,
          week = week,
          week_minus_1 = week_minus_1,
          week_minus_2 = week_minus_2,
          week_minus_3 = week_minus_3,
          borders = borders,
          colorblind_friendly = colorblind_friendly
        )
        
        cat("\n\n")
        
        # Call cum_ci_plot with explicit parameters
        cum_ci_plot(
          pivotName = field_name,
          ci_quadrant_data = CI_quadrant,
          plot_type = ci_plot_type,
          facet_on = facet_by_season,
          x_unit = x_axis_unit,
          colorblind_friendly = colorblind_friendly
        )
        
        cat("\n\n")
        
      }, error = function(e) {
        safe_log(paste("Error generating plots for field", field_name, ":", e$message), "ERROR")
        cat("\\newpage\n\n")
        cat("# Error generating plots for field ", field_name, "\n\n")
        cat("Data not available for visualization\n\n")
      })
    }
  }, error = function(e) {
    safe_log(paste("Error in field visualization section:", e$message), "ERROR")
    cat("Error generating field plots. See log for details.\n\n")
  })
} else {
  cat("Field visualization data not available. Required data objects are missing.\n\n")
  cat("Please ensure scripts 02 (CI extraction) and 03 (growth model) have been run successfully.\n\n")
}
```

---

\newpage

## Detailed Field Summary Table

The following table provides a comprehensive overview of all monitored fields with their key performance metrics.

```{r detailed_field_table, echo=FALSE}
# Clean up the field details table - remove sub field column and round numeric values
field_details_clean <- field_details_table %>%
  select(-`Sub Field`) %>%  # Remove Sub Field column
  mutate(
    `Mean CI` = round(`Mean CI`, 2),    # Round to 2 decimal places
    `CV Value` = round(`CV Value`, 2)   # Round to 2 decimal places
  )

# Display the cleaned field table
kable(field_details_clean,
      caption = "Detailed Field Performance Summary")
```

---

## Report Metadata

```{r report_metadata, echo=FALSE}
metadata_info <- data.frame(
  Metric = c("Report Generated", "Data Source", "Analysis Period", "Total Fields", 
             "KPI Calculation", "Next Update"),
  Value = c(
    format(Sys.time(), "%Y-%m-%d %H:%M:%S"),
    paste("Project", toupper(project_dir)),
    paste("Week", current_week, "of", year),
    ifelse(exists("field_boundaries_sf"), nrow(field_boundaries_sf), "Unknown"),
    ifelse(kpi_files_exist, "✓ Current", "⚠ Needs Update"),
    "Next Wednesday"
  )
)

kable(metadata_info,
      caption = "Report Metadata",
      col.names = c("Metric", "Value"))
```

---

*This report was automatically generated by the SmartCane monitoring system. For questions or additional analysis, please contact the technical team.*