Section 2 Data Cleaning and Processing

This page covers Section II of the methodology covered in my report.

First, load all of the libraries required for this code:

2.1 Heart rate data

To read in all of the .TCX files, first create a list of all filenames in the ‘data’ directory with the appropriate extension.

This function will then perform the necessary processing for each .TCX file. We will read in each .TCX file in XML format, then convert it to a dataframe in R. We will make sure that each column is the appropriate data type and then convert the dataframe to a MoveStack object.

We will apply this function to all of our data files. Then we will add all of the data together to create a single object.

Next select the columns of interest from our data and write them to a .CSV file for us to take a look at manually.

Here is a subset of what the data now looks like:

##   test3$TimeConverted    X_coord  Y_coord  hr                time
## 1 2019-12-05 23:11:59 -0.1273531 51.50747 130 2019-12-05 23:11:59
## 2 2019-12-05 23:12:02 -0.1273531 51.50747 129 2019-12-05 23:12:02
## 3 2019-12-05 23:12:03 -0.1273531 51.50747 129 2019-12-05 23:12:03
## 4 2019-12-05 23:12:04 -0.1273501 51.50747 129 2019-12-05 23:12:04
## 5 2019-12-05 23:12:05 -0.1273423 51.50747 127 2019-12-05 23:12:05
## 6 2019-12-05 23:12:06 -0.1273263 51.50747 127 2019-12-05 23:12:06

2.2 Creating a continuous heart rate layer

We will now process the cleaned heart rate points to create a continuous layer of anxiety levels. This processing is done using QGIS, and so a model with predefined steps and parameters has been provided: Link to model

The input for this model is a vector layer with points corresponding to each heart rate measurement. The QGIS ‘Points Layer from Table’ function can be used to read in the .CSV file generated in the previous step and create this vector layer.

The QGIS processing model performs the following functions:

  1. Interpolates the points to create a continuous surface of heart rate values
  2. Creates a buffer around all point locations.
  3. Clips the interpolated layer by the buffer to enclose the region of heart rate measurement.
  4. Smooths the interpolated layer.

The model outputs a raster layer of continuous heart rate values. This file should be saved as a GeoTIFF for future visualization.

2.3 Personal comments

We now need to read in the personal comments CSV file, stored in the same folder as our heart rate data.

Here is what this data looks like:

##                                                                 ï..Note
## 1                                          Wow, very crowded sidewalk. 
## 2                         Got stuck in the middle of the intersection. 
## 3          Tried to cross the street but I didn't see that car coming. 
## 4 I think this is the area where people have reported getting harassed.
## 5 I always get stuck in the middle of the intersection on this street. 
## 6                         There are some bikes moving very quickly here
##          Date_time
## 1 2019-12-11 17:22
## 2 2019-12-11 17:23
## 3 2019-12-11 17:32
## 4 2019-12-11 17:33
## 5 2019-12-13 13:52
## 6 2019-12-04 13:58

Our next task is to attach coordinates to each note, by joining through time with the heart rate data, as processed previously.

Now we have our coordinates:

##    test3$TimeConverted    X_coord  Y_coord  hr                time
## 1: 2019-12-04 13:58:00 -0.1263700 51.52482 130 2019-12-04 13:58:00
## 2: 2019-12-04 13:59:00 -0.1274354 51.52488 129 2019-12-04 13:59:00
## 3: 2019-12-04 14:04:53 -0.1323954 51.52330 124 2019-12-04 19:11:00
## 4: 2019-12-04 14:04:53 -0.1323954 51.52330 124 2019-12-04 19:15:00
## 5: 2019-12-05 11:31:00 -0.1249467 51.52490 140 2019-12-05 11:31:00
## 6: 2019-12-05 18:52:00 -0.1184381 51.52405 109 2019-12-05 18:52:00
##                                                                              ï..Note
## 1:                                     There are some bikes moving very quickly here
## 2:                                                     This park looks very peaceful
## 3: Still haven't gotten the hang of looking a different way when crossing the street
## 4:                                             It's nice not to see any cars in here
## 5:                              Almost didn't see that car coming around the corner 
## 6:                         The pedestrian scale of this street is really comfortable
##           Date_time
## 1: 2019-12-04 13:58
## 2: 2019-12-04 13:59
## 3: 2019-12-04 19:11
## 4: 2019-12-04 19:15
## 5: 2019-12-05 11:31
## 6: 2019-12-05 18:52