QGIS: Fieldwork Data Collection

Setting up

This is an intermediate level tutorial. Before completing this tutorial, we recommend our QGIS: Introduction to Mapping tutorial. This tutorial is designed for QGIS 3.40. If you need to install it on your computer, go to the QGIS website.

We will start as always by creating a good folder structure to work within. This folder is where our project, our data, and creations will live. Folder structure is very important for keeping your data tidy, as well as for ease of sharing your project with others. You simply need to zip the project folder if you need to share the whole thing.

• Open QGIS and create a new project with Project > New.

• Let’s now save our project: Project > Save.

• Create a new folder, let’s call it “qgis_fieldwork”.

• Inside that folder, create these folders:

  • “data” - for all the data we will use to make our maps, split into:

    • “raw” - raw data from your research or the internet

    • “processed” - any data you’ve modified

  • “output” - for any maps or images we export

  • “DCIM” - this is a new one, it’s for storing images from our phone!

  • “temp” - this isn’t necessary, but when you’re playing around and testing, it stops things getting messy.

• Finally, let’s save our .qgz project file here, named “qgis_fieldwork_map.qgz”

Your .qgz file should always be in the highest level folder, so it’s only looking down into folders for data, not back out.

This might seem necessary now, but things quickly get out of control and hard to find if you don’t have a good folder structure.

Let’s finally add an OpenStreetMap basemap to locate ourselves on the globe:

• Browser panel > XYZ Tiles > OpenStreetMap (double-click, or drag and drop into the Layers panel).

Install QField and Avenza Maps on your phone

QField and Avenza Maps are a mobile based offline mapping tools you can use on your smartphone.

QField

QField plugs directly in to QGIS, and allows you to link a QGIS project with your mobile phone for data collection.

Android Download iOS Download

Avenza

For Avenza Maps you just need to add a GeoPDF and you can collect data on it. It’s less functional than QField, but it’s a great fallback tool. Android Download iOS Download

There are alternatives out there, such as Input (MerginMaps).

Get some data

We’re going to explore a number of different online spatial data repositories.

DEM

A Digital Elevation Model (DEM) is a common example of raster data, i.e. grid data that contains a value in each cell (a bit like the pixels in a coloured picture).

For this tutorial, we are using a DEM sourced from ELVIS - Geoscience Australia’s ELeVation Information System.

• Go to http://www.ga.gov.au/elvis/
• Search for “St Lucia” in the Location Search search box and select the first result
• Click Order Data
• Choose “Draw” and “Box” and then click the Draw button
• Click on the map to start drawing a rectangle around your desired area
• Then click Search
• The right panel will show you all the different datasets available in that area
• We want the QLD Government Digital Elevation Model 1 Metre, click the down arrow on the right
• As you hover over the different options, they will highlight a red box on the map, click the tick box and select all that overlap the area you’re interested in (note that there may be data from different years here)
• When you have the data you want, scroll to the bottom of the Order Data window
• Select your industry, enter your email, tick the reCAPTCHA, and click the Order Datasets button
• You should receive an email within 5 minutes, download the files from the link in the email, extract the folder to your project folder, and add them to your map.

Aerial Imagery

There are a few places you can aquire aerial photography, today we will look at two sources, one is freely available Government Data, the other is accessible from using your UQ credentials.

As a UQ student, you also have access to very high resoltuion imagery from NearMap. You can even access an array of imagery going back in time.

• Go to the UQ NearMap Portal
• Enter your UQ Student (@student.uq.edu.au) or Staff (@uq.edu.au) email address, with the appropriate domain selected. Click “Invite me”.
• You should receive an email soon after, click the “Accept Invitation” button, and go through the account setup process.
• Go to Login and enter your email address, click “Next” and enter your password.
• From the top right select MapBrowser.
• Type “St Lucia, QLD” in the search box, press enter
• You can click the date down the bottom to look at different snapshots in time, and even compare maps side-by-side.
• To save imagery from NearMap, simply click the “Exports” button on the left handside (it is an image icon)
• From the menu that appears change the “Export type” to “Georeferenced image”
• From “Projection” choose GDA2020 / MGA zone 56
• You can increase and decrease the size of the bounding box by adjusting its corners, a smaller box means you can have a finer resolution, down to 0.075m. If we select all of UQ St Lucia in one go, the highest resolution we can have is 0.299m.
• Once you’re happy with your selection click Download Files
• Move the downloaded zip file to your project folder, and open them in QGIS

If you want more aerial images or other spatial files, it’s worth looking at a range of government sources. Your particular study area may have maps available such as the Sunshine Coast MyMaps service, CSIRO’s Norfolk Island Data Portal or other CSIRO spatial products.

Lot Plans

You can access a wide variety of QLD Government Data, including Spatial Data such as lot plans and vegetation maps, from QLD Spatial. There are three ways to access this data. You can download all of it, you can select a portion for download, or you can live load it into your project. Today we download the files covering our project area.

• To access data from QLD Spatial go to https://qldspatial.information.qld.gov.au/

• Search for “property boundaries”

• Scroll down to “Property boundaries Queensland” and click Add to my list

• Click My List from the top menu

• Click View/extract in map

• Under Extractable Data, click the box next to Property boundaries Queensland, it will become green

• Click Extract/download

• Click Choose an area

• Select BRISBANE CITY from the Select LGA option

• Choose GeoPackage 1.0 from Select output format

• Enter your Email Address

• Accept the Terms and Conditions

• Click Extract/Download

• You can can also just download the preprepared data here, just press the three dots and download.

• Move the downloaded zip file to your project raw data folder, and open the shapefile in QGIS

Queensland Globe

Another way to access QSpatial Data is using the Queensland Globe portal.

• To access data from Queensland Globe go to https://qldglobe.information.qld.gov.au
• Accept the T&Cs.
• Click Search, Select Locality (Suburb) within a Local Government Area, and search for the location you want and select it from the list.
• Click Layers, click Add Layers + here you can scroll through and filter different layer types
• We want to tick the box next to Planning Cadastre > Land Parcels > Land Parcel (you may need to zoom in to see certain layers)
• To export data click the Wrench Icon in the top right, and then click the Identify icon (i)
• Use the triangular Identify Polygon tool to select and area of interest. Double click when you’ve finished selecting your area.
• The Layers menu will now show your selection. You can download all, or sections, of your selection.
• I will choose Land Parcel and then Download as a shp file.
• Note: You may need to disable other layers for this to work correctly. I needed to turn off the Transportation layer to prevent roads from being included in my selection.

Import and process our data

If you have saved your data into your data folder within your project folder qgis_mobile, it should appear in Project Home. You can import it into your project’s layers by double-clicking on the data there.

Create a buffer to clip our data

We now want to only keep data that is relevant to the St Lucia campus. To achieve that, we need a mask to clip data.

We first need to select the lots that are relevant to us: - Use the “Select Feature by area or single click” (in the toolbar) - Click on the lots one by one (or draw a rectangle around them) with the Shift key pressed.

We can now create a buffer:

• Go to Vector > Geoprocessing Tools > Buffer

  • There are many settings we can change to edit the shape of the buffer, but we will leave most of these as the defaults for now.

• Choose “Brisbane_lot_plans” as the Input Layer, and tick the Selected features only box

• We next want to choose distance that we want to create a buffer away from our vector.

  • You’ll note that the distance is currently in degrees. This is because the layer projection is EPSG:4283 GDA94, a geographic coordinate system. This might cause issues.

• If you click Run. Your buffer won’t look very good.

  • We first need to convert our layer to a local projected coordinate system.

Reproject Our Data

Let’s reproject the layer for better results.

We have two options here. We can either reproject the whole layer as we did in the previous workshop, OR we can export our selection and reproject it as we export. Let’s do the latter.

• With our desired area still selected, right click on the lot plan layer Export > Save Selected Features As...
• Set the Format to GeoPackage
• Click the three dots ... next to the File name field, navigate to the project’s processed data folder, and type in UQ_lot_plan and click Save
• Set the CRS to EPSG:7856 - GDA2020 / MGA zone 56
• Click OK

The new layer will look the same, but it now has a different projection.

Buffer again

Let’s create that buffer again.

• Select the relevant lots
• Go to Vector > Geoprocessing Tools > Buffer
• This time choose UQ_lot_plan as the Input Layer
• You’ll notice the distance is now in metres. For this instance, I want a larger buffer of 50m.
• When you click Run, your buffer should be a good distance away from the original boundary.

Dissolve Polygons

Now we can Dissolve those polygons together:

• Go to Vector > Geoprocessing Tools > Dissolve
• Choose “Buffered” as the Input Layer.
• Let’s save this one to file, rather than a temporary file.
  • Click “…” next to “[Create Temporary Layer]”,
  • Then “Save to file”, and save it as UQ Boundary to your products folder.

Note the Buffer tool has a “dissolve” option, but it’s important to know that there are many tools available.

Combine our DEMs

See the visible line between the raster tiles? That is because the separate raster files have different maximum and minimum values, so use different shades for different elevations. We have to merge them to make sure they use the same colour scale.

To do that, we use the Raster > Miscellaneous > Merge... tool to create one single layer from them.

• First, select all DEM layers for the “Input layers”
• Make sure the option “Place each input file into a separate band” is unchecked, as we want to end up with one single-band layer
• Click “Run”

You will need to have Saga installed for this to work.

We can now remove the original DEM raster files.

Clip the DEM

We now use Raster > Extraction > Clip Raster by Extent to focus on our area of interest.

• Next to “Clipping extent” click ... > Calculate from Layer > UQ_Boundary

• Scroll down to “Clipped (extent)” and click “…” next to [Create Temporary Layer]“, then”Save to file”, and save it as UQ_DEM to your project folder.

• Click Run

Another way to clip would be to use Mask Layer, but I prefer Extent, as the file size is the same, but you get more interesting data.

Note

• Make sure the virtual raster output DEM is selected in the Input layer

• Navigate to Raster > Extraction > Clip Raster by Mask Layer

• Set the Mask Layer to “UQ_Boundary”

• Click Run

If you don’t save to file directly, remember two things:

• rename your clipped layer so it is more descriptive than the generic “Clipped (extent)”
• you are currently using a temporary, scratch layer. It will be discarded if you exit QGIS. It is very useful for temporary intermediate files, but it can be safer to save copies of your intermediate data while you work, just in case! You can right-click on the layer and use Export > Save As..

Save your project

It’s not essential at this stage, it’s just a good reminder to save your work regularly so you don’t lose things when things go wrong.

Clip the Aerial Imagery

We can repeat the same process as above to clip the Aerial Imagery.

Now, clip the converted aerial imagery:

• Go to Raster > Extraction > Clip Raster by Extent
• Make sure St_Lucia_Aerial is selected in the Input layer
• Set the Mask Layer to “UQ_Boundary”
• You may need to define the Source and Target CRS to ensure that it all runs smoothly
• Scroll down to “Clipped (mask)” and click “…” next to [Create Temporary Layer]“, then”Save to file”, and save it as St_Lucia_Aerial_clipped to your project folder.

We can now clean up our project by removing the original and intermediate layers we won’t need.

Change the symbology

We can style our DEM with a terrain colour palette:

• double-click on the clipped DEM layer
• go to the “Symbology” tab
• change the Render type to “Singleband pseudocolor”
• by default, it uses the min/max values, which is what we want
• we can change the “Color ramp” to something more suitable with the drop-down menu and Create new color ramp... > Catalog: cpt-city > Topography > Elevation, for example.

Add a hillshade

Adding a hillshade makes your visualisation of elevation more readable and visually pleasing by giving an artificial lighting look to your map.

• Go to Raster > Analysis > Hillshade....
• Ensure the Input Layer is your clipped DEM “St_Lucia_DEM”
• Scroll down to “Hillshade” and click “…” next to Create Temporary Layer, then “Save to file”, and save it as St_Lucia_DEM_Hillshade to your project folder.
• Click Run

Create Contours

Adding a contour makes your elevation even more evident and can also be used to quickly show elevation on other layers

• Go to Raster > Extraction > Contour....
• Ensure the Input Layer is your clipped DEM “St_Lucia_DEM”
• For Interval between contour lines, the default is 10 m, which won’t be greatly noticeable at UQ where the highest point is 25 m. However, the finer the contour lines, the longer it will take to process. I will choose 5 m.
• Scroll down to “Contours” and click “…” next to [Create Temporary Layer]“, then”Save to file”, and save it as St_Lucia_Contours_5m to your project folder.
• Click Run

Create a new layer for our data collection

This is a very important step, as you cannot create new layers in QField, so we need this to record the data we collect.

• Go to Layer > Create Layer > New GeoPackage Layer...
• Click the three dots ... next to the Database field, navigate to the project’s processed data folder, and type in fieldwork_data and click Save
• For Geometry type choose MultiPoint
• Set the CRS to EPSG:7844 - GDA2020
• Under New Field we will add our layer attributes:
• Let’s call the point name “Name”
  • Name: Name
  • Type: Text (string)
  • Click Add to Fields List
• Description
  • Name: Description
  • Type: Text (string)
  • Click Add to Fields List
• Date and Time
  • Name: DateTime
  • Type: Date & Time
  • Click Add to Fields List
• Latitude
  • Name: Lat
  • Type: Decimal (double)
  • Click Add to Fields List
• Longtitude
  • Name: Long
  • Type: Decimal (double)
  • Click Add to Fields List
• Click OK

Add another field

It would be nice to be able to take photos when we collect our data, let’s add another field.

• Double click on the new fieldwork_data layer
• Click on the Fields tab
• Click the yellow pencil button to Toggle Editing Mode
• Click the now visible yellow New field button
  • Name: Photo
  • Type: Text (string)
  • Click OK
• Click the yellow pencil button to Toggle Editing Mode
• Click Save
• Click OK

Prepare the Attributes Form

Now that we have a layer with all the desired fields, we can now edit the layer form to customise how things are entered.

• Double click on the new fieldwork_data layer
• Click on the Attributes Form tab
• At the very top of the window, click Autogenerate and change it to Drag and Drop Designer

On the left you will see all of our Fields, we can drag and drop these into the Form Layout section - this section lays out how our form will look when we enter new data.

• Drag and Drop the Photo Field to the Form Layout so that it is sitting above DateTime
• Click fid and press the red minus button - we want to keep the field, but don’t need to see this in our form
• Click Name
  • Under Constraints tick Not Null and Enforce not null constraint
  • We don’t need to change anything else for Name, we just want to make it mandatory
• Click Photo
  • Under Widget Type click Text Edit and change it to Attachment
  • Under Path change Default path to @project_home + ‘/DCIM’
    • Change Absolute Path to Relative to Project Path
  • Tick **Use a hyperlink for document path (read-only)
  • Under Integrated Document Viewer change Type from No Content to Image
  • Click DateTime
    • Under General untick Editable - we don’t want to edit this field
    • Scroll to the bottom, under Defaults change the Default value to $now
    • If this worked, there will be a date and time next to Preview
  • Click Lat
    • Under General untick Editable - we don’t want to edit this field
    • Scroll to the bottom, under Defaults change the Default value to $y
    • The Preview will show NULL, but that’s okay, we don’t have data yet
      
  • Click Long
    • Under General untick Editable - we don’t want to edit this field
    • Scroll to the bottom, under Defaults change the Default value to $x
      
• Click OK

We should now have a functional form, let’s test it in QGIS before we send it to our phones

• Click the yellow pencil button to Toggle Editing on
• Click the green dots (or press Ctrl + .) to Add Point Feature
• Click on the map to add a point
• The Name should be mandatory, the Description editable, Photo have an add option, DateTime should have the current date and time, and Lat and Long should have numbers like “-27.494066205142065” and “153.0153703956312”
• Click Cancel
• Click the yellow pencil button to Toggle Editing off

We’re ready to export our project to QField.

• Navigate to Plugins > Manage and Install Plugins...
• Search for “QField”
• Click on QField Sync
• Click Install Plugin
• Once it has installed, click Close

You can now access the QField Sync Plugin through the menu.

• Navigate to Plugins > QFieldSync > Package for QField
• If you get a Message window popup, click Next
• Give your project an appropriate title qgis_fieldwork
• Under Packaged Project Filename navigate to your outputs folder and create a new folder called UQ_Fieldwork, open that folder and click Save
• Under Advanced untick data and outputs
• Click Create

We now need to zip this folder and send it to our phone!

• Navigate to your Outputs folder, right click on the UQ_Fieldwork folder and compress it to a zip file.
• Send this file to your phone via email, cloud, or using a cable.

Using QField

• Open QField
• Skip any popups
• Tap Open local file
• Tap the green plus button in the bottom right
• Tap Import project from ZIP
• Navigate to your zip file. In my case, I had to go to the top left menu, and tap on Downloads
• The file should open in QField. You may need to tap on the outer folder, and then the Project

Feedback

Please visit our website to provide feedback and find upcoming training courses we have on offer.

Exporting to Avenza Maps

Note

Click on Show Layout Manager in the toolbar or use Project > Layout Manager. Create a new layout called “Avenza”. We can now see the Layout window.

Normally we would add many elements to our layout if we were exporting it for print such as the map, a legend, a scale bar, a north arrow…

In this case however, we are simply interested in our map. Let’s add the map to the canvas:

• Go to the Layout tab, scroll down to ‘Resize Layout to Content’, click ‘Resize layout’
• Before we export, let’s turn off any layers we aren’t using in QGIS to save space
• Click the Refresh View button up the top
• Now we are ready to export.
• Go to ‘Layout > Export as PDF…’ and save your map.
• The ‘PDF Export Options’ window will open
• Tick the ‘Create Geospatial PDF (GeoPDF)’ box
• Click ‘Save’

You can repeat this process with the DEM and Hillshade to export out another kind of map.

Now you simply need to export the pdf file(s) to your phone. You can email it, send it through the cloud, or transfer it using a cable.

When you first open Avenza Maps it will ask you to create an account, but you can import up to three maps without doing so, you can avoid creating an account by selecting the ‘x’ in the top right corner. * Allow Avenza Maps to access your device location * Select the orange ‘+’ in the bottom right and select ‘Download or import a map’ * Choose ‘From Storage Locations’ (if requested, give Avenza Maps the permissions to access your files) * Do the same for the other map, if you created one. * Once it has been imported, tap on the map. * You can now move the map around with your finger, and pinching to zoom. * Tapping the placemark icon in the bottom left will add a placemark in the middle of the crosshairs. * Tapping the 3 dots in the bottom right will allow you to add GPS tracking and draw and measure distances.