Matera - Day 4

The course gets split up in basic R and advanced R users for the morning.

Session 1 - R basics

• R has some bash capabilities and replicates all data which is being processed internally

• R stores variables in their data represention which is kept in the RAM; rather than just keeping the “formula” (like Python does)

• R is not really connected to bash; “enter R, close the door to bash”

• sudo needed for installing packages

rm(a) - remove variable
gc() - clean RAM (garbage collector)
?<command> - get help on command (with examples)
q() - quite R
system("pwd") - run system command
data.frame=read.table("filename") - read something from outside into R into data frame 
str(data.frame) - show structure
$ - indicates another level, e.g. landuse04$landuse
head(data.frame) - show head of data frame
object.size(dem) - show byte size of data frame
dem$X=as.character(dem$X) - change data type of 'X' in 'dem' data frame to character
save(landuse04, file="~/landuse2004.Rdata") - save data frame as file
save.image() - save whole workspace
load("~landuse2004.Rdata")
rm(list = ls()) - remove everything in workspace
plot(landuse$fallow.Fallow, landuse$vineyard.Vineyards) - crude plotting
landuse[1:3 , 3:10] - access data via indices; first value pair = rows; second value pair = columns

• get information & inspiration for graphing/plotting: R graph gallery

install.packages("raster")
library(raster)
myinput=raster("/home/user/ost4sem/exercise/basic_adv_gdalogr/input.tif")
plot(myinput)
    -- install raster package, load it, load file and plot the raster
    -- raster is being kept in file instead of memory
    
@ - sub-level indicator for raster images    

Session 2 - conference call

• Victoria O’Brien - an introduction to sentinel satellite data

• Sentinel 1 - radar data, continously collecting data, 6 day cyle

• Sentinel 2 - visible data, multispectral, resolution: 10, 20, 60 meters, 5 day cycle

• data is kept by multiple organizations

  • access via browser; e.g. copernicus open access hub (registration needed)
  • access via tiles; use KML to get the organization of the tiles; get them on AWS
  • also a CLI tool available: sentinalsat

• ESA SNAP - software for exploring, processing and classifying remote sensing data

• copernicus emergency management system; e.g. rapid flood mapping, forest fires, etc.

Session 3 - R basics / distribution modelling

• BioClim as a way to incorporate multiple important indicators for analysis

rbind(presence,absence) - join two tables   
table(points$PA) - count occurences of attribute
na - handle missing values (omit, fail, etc.)
c - combine values into list or vector

• sp-package is quite useful for spatial operations

• general idea for distribution modeling (sort of): get predictor, standardize, make tables, construct model with predictors

Session 4 - GRASS basics

• bash is available in GRASS text mode

grass70 -text ~/ost4sem/grassdb/europe/PERMANENT - start GRASS in textmode and load location 'europe' in 'grassdb'
r.info --ui - runs the r.info function (info about a layer) and starts the GUI dialogue for it

• more interesting commands:

g.copy rast=potveg_ita@Vmodel,pvegita - copy within GRASS
g.remove -f type=raster name=pvegita - remove raster dataset
g.region -p - get current region 
g.region n=6015390 e=5676400 s=3303955 w=3876180 res 1000 save=scandinavia --overwrite - set new region
g.region res=20000 -p - change resolution
g.gui tcltk - bring up GUI (possible arguments for GUI wxpython,text,gtext on this particular machine)

g.list type=rast -p - list all raster maps (-p for pretty printing)

# We can open a monitor and display a raster 
g.region rast=fnfpc
d.mon start=x0
d.rast fnfpc

# and do the same thing for theother maps in different monitors
d.mon start=x1    
d.rast fnfpc_alpine10k 

# get input into GRASS
r.in.gdal input=~/ost4sem/exercise/basic_adv_grass/inputs/lc_cor2000/hdr.adf  output=landcover 

Session 5 - remote sensing & machine learning

• Tom Jones, EO specialist, Catapult Satellite Application

• current time is disruptive in terms of satellite imaging; many factors are changing quite dramatically, so fast moving technology companies have real adavantage

• since 2013 rapid increase in downloads of Landsat scenes

• remotely monitored road conditions are one example for future applications

• OS software in the field: OpenCV, SciLab, RGISLib, SNAP, GDAL, scikit learn

• Proprietary software: IDL, eCognition, ArcGIS, FME,

• random bits: Jupyter notebooks, sklearn python module, ARCSI CLI pre-processing for remote sensing images, ShepherdSegmentation algorithm for clumping, tuiview