Field Data Collection
This last session will look at using QField to collect data in the field. We can turn a QGIS project into a QField project we can use on our phones.
Folder Setup
We need to create two new folders in our Enoggera_Windfarm_26 folder
- “DCIM” - this is for storing images from our phone!
- “EH_Fieldwork” - we need this one to export our project to QField
Install QField
QField plugs directly in to QGIS, and allows you to link a QGIS project with your mobile phone for data collection.
Buffer Data
As we’ve seen, buffers are a simple, but powerful, tool that we can use to extend the borders of our vector data. It’s useful for polygons, lines, and points.
Perhaps we need to keep 10m from the Project Area boundaries when doing our fieldwork. A buffer can help us there.
Polygons to Lines
To buffer either side of the edges of our boundaries, we need to convert them from Polygons to Lines.
- Go to
Vector > Geometry Tools > Polygons to Lines - Set the
Input Layerto Project Area - Click
OK
This will create a temporary file called Lines, you can choose to save it instead, but that is optional.
Buffer the Lines
Go to
Vector > Geoprocessing Tools > BufferChoose “Lines” as the
Input LayerSet the
Distanceto 10 metres- We can also make the buffer squared if we want, but we will keep it with the rounded default
Tick the
Dissolve resultbox- This will merge the polygons from our output, making for a cleaner look. The Dissolve tool also exists as a standalone tool.
Click the three dots
...next to the Buffered field, navigate to the project’s processed data folder, and type in EH_Boundary_Buffer and clickSaveClick
Run
Style the Boundary buffer polygon
- Open the
Layer Stylingpanel by pressing F7 (or fn + F7) - Select the EH_Boundary_Buffer layer from either the
Layer Stylingpanel, or theLayerspanel. - Under
Fill, clickSimple Fill- Click the dropdown next to
Fill colorand select a colour that reminds us not to go there, like orange- Change the
Opacityto something around 30%
- Change the
- Click the dropdown next to
Save your project
It’s not essential at this stage, it’s just a good reminder to save your work regularly so you don’t lose things when things go wrong.
Layer for Data Collection
This is a very important step, as you cannot create new layers in QField, so we need this to record the data we collect.
- Go to
Layer > Create Layer > New GeoPackage Layer... - Click the three dots
...next to the Database field, navigate to the project’s processed data folder, and type in collected_data and clickSave - For
Geometry typechoose MultiPoint - Set the CRS to
EPSG:4326 - WGS 84 - Under New Field we will add our layer attributes:
- Let’s call the point name “ID”
Name: IDType: Text (string)- Click
Add to Fields List
- Description
Name: DescriptionType: Text (string)- Click
Add to Fields List
- Category
Name: CategoryType: Text (string)- Click
Add to Fields List
- Date and Time
Name: DateTimeType: Date & Time- Click
Add to Fields List
- Latitude
Name: LatType: Decimal (double)- Click
Add to Fields List
- Longtitude
Name: LongType: Decimal (double)- Click
Add to Fields List
- Click
OK
You might wonder why we’ve just chosen WGS84 after seeing the issues with it earlier. Placing points with this projection is less of an issue than it is for doing measurements and analysis. For today we’re choosing WGS84 so that our Lat and Long values are in a format most people expect: WGS84. If we chose a Projected Coordinate System, it wouldn’t give us location in degrees Lat: -27.4705, Lon: 153.0260, it would give us a location meters Easting: 500,000 Northing: 6,900,000 We could choose a local Geographic Coordinate System like EPSG:7844 - GDA2020, but again, most people will assume your values are in EPSG:4326 - WGS 84, so that’s a safe bet unless your group/company has a preferred option.
Always check and transform to a common CRS before analysis.
Add Another Field
It would be nice to be able to take photos when we collect our data, let’s add another field.
- Double click on the new fieldwork_data layer
- Click on the
Fieldstab - Click the yellow pencil button to Toggle Editing Mode
- Click the now visible yellow New field button
Name: PhotoType: Text (string)- Click
OK
- Click the yellow pencil button to Toggle Editing Mode
- Click
Save - Click
OK
Attributes Form Preparation
Now that we have a layer with all the desired fields, we can now edit the layer form to customise how things are entered.
- Double click on the new collected_data layer
- Click on the
Attributes Formtab - At the very top of the window, click
Autogenerateand change it toDrag and Drop Designer
On the left you will see all of our Fields, we can drag and drop these into the Form Layout section - this section lays out how our form will look when we enter new data.
Drag and Drop the Photo Field to the
Form Layoutso that it is sitting above DateTimeClick fid and press the red minus button - we want to keep the field, but don’t need to see this in our form
Click ID
- Under Constraints tick
Not NullandEnforce not null constraint - We don’t need to change anything else for ID, we just want to make it mandatory
- Under Constraints tick
Click Category
- Under Widget Type click
Text Editand change it toValue Map - Add the following to
ValuesandDescriptions:EXandExcellentGDandGoodPRandPoorOTandOther- It is good practice to have an Other field if you’re using a “Not Null” constraint, as it allows the user to note when something doesn’t meet the given Categories.
- Under Constraints tick
Not NullandEnforce not null constraint
- Under Widget Type click
Click Photo
- Under Widget Type click
Text Editand change it toAttachment - Under Path change
Default pathto @project_home + ‘/DCIM’- Change
Absolute PathtoRelative to Project Path
- Change
- Tick Use a hyperlink for document path (read-only)
- Under Integrated Document Viewer change Type from
No ContenttoImage
- Under Widget Type click
Click DateTime
- Under General untick
Editable- we don’t want to edit this field - Scroll to the bottom, under Defaults change the
Default valueto $now - If this worked, there will be a date and time next to
Preview
- Under General untick
Click Lat
- Under General untick
Editable- we don’t want to edit this field - Scroll to the bottom, under Defaults change the
Default valueto $y - The
Previewwill show NULL, but that’s okay, we don’t have data yet
- Under General untick
Click Long - Under General untick
Editable- we don’t want to edit this field - Scroll to the bottom, under Defaults change theDefault valueto $x
Click
OK
We should now have a functional form, let’s test it in QGIS before we send it to our phones
- Click the yellow pencil button to Toggle Editing on
- Click the green dots (or press Ctrl + .) to Add Point Feature
- Click on the map to add a point
- The Name should be mandatory, the Description editable, Photo have an add option, DateTime should have the current date and time, and Lat and Long should have numbers like “-27.494066205142065” and “153.0153703956312”
- Click
Cancel - Click the yellow pencil button to Toggle Editing off
Tracks Layer
This is another important step, as you need to create your tracking layer before you export to QField
- Go to
Layer > Create Layer > New GeoPackage Layer... - Click the three dots
...next to the Database field, navigate to the project’s processed data folder, and type in tracks and clickSave - For
Geometry typechoose MultiLine - Set the CRS to
EPSG:23856 - GDA94 / MGA zone 56 - Under New Field we will add just one layer attribute:
- Let’s call the point name “DateTime”
Name: DateTimeType: Date & Time- Click
Add to Fields List
Tracks Attributes Form Preparation
Let’s prepare the tracks DateTime as we did with the collected_data - Double click on the new tracks layer - Click on the Attributes Form tab - At the very top of the window, click Autogenerate and change it to Drag and Drop Designer
On the left you will see all of our Fields, we can drag and drop these into the Form Layout section - this section lays out how our form will look when we enter new data.
- Click fid and press the red minus button - we want to keep the field, but don’t need to see this in our form
- Click DateTime
- Under General untick
Editable- we don’t want to edit this field - Scroll to the bottom, under Defaults change the
Default valueto $now - If this worked, there will be a date and time next to
Preview
- Under General untick
- Click
OK
QField
We’re ready to export our project to QField.
Install QField Plugin
- Navigate to
Plugins > Manage and Install Plugins... - Search for “QField”
- Click on QField Sync
- Click
Install Plugin - Once it has installed, click
Close
You can now access the QField Sync Plugin through the menu.
Prepare Data for QField
At this point we need to clean up our Layers. You don’t have to remove everything, but it’s worth deciding what layers you want to remove, and what scratch layers you want to make permanent.
Hide any layers you want to keep, but don’t want to export to QField
Make sure your layer order makes sense, I recommend the following layers visible in this order:
- collected_data
- tracks
- EH_Boundary_Buffer
- EH_Contours_10m
- EH_DEM_5m
- EH_DEM_5m_Hillshade
- World Imagery Tile
Export to QField
- Navigate to
Plugins > QFieldSync > Package for QField - If you get a Message window popup, click
Next - Give your project an appropriate title EH_Fieldwork
- Under Packaged Project Filename navigate to your EH_Fieldwork folder and click
Save - Under Advanced untick every folder except
DCIM- We want a copy of this folder for our photos to be saved to. The rest of our data will be packaged into the QField folder.
- At the bottom of the window click
Configure current project...- Click the
Cable Exporttab (if we were using QFieldCloud, we would click that option now instead) - Click the
Toggle Layerseye symbol on the right- In the
Actioncolumn changeCopytoRemove from projectfor any layers you don’t want to copy to QField.
- In the
- Scroll down and tick Geofencing
- Set
Geofencing areas layerto EH_Boundary_Buffer
- Set
- Click
OK
- Click the
- Click
Create
We now need to zip the QField folder and send it to our phone!
- Navigate to your Project folder, right click on the EH_Fieldwork folder and compress it to a zip file.
- Send this file to your phone via email, cloud, or using a cable.
Opening the QField Project
Android
- Open QField
- Skip any popups
- Tap
Open local file - Tap the green plus button in the bottom right
- Tap
Import project from ZIP - Navigate to your zip file. In my case, I had to go to the top left menu, and tap on Downloads
- The file should open in QField. You may need to tap on the outer folder, and then the Project itself.
iOS
- Open the zip file and extract it to:
On My iPhone/QField/Imported Projects - Open QField
- Skip any popups
- Tap
Open local file - Tap
Imported projects - Open the EH_Fieldwork folder until you reach the EH_Fieldwork Project file.
- The project should open in QField.
You can now access your map.
Using QField
- Tap the menu in the top right to see your layers.
- Tap the eye to hide or unhide layers
- If you tap on a layer, and then tap the pencil icon in the bottom right, you can edit a layer.
- Let’s do this for our data_collected layer
- Tap on the map on the right to return to the map, your cursor will now be a cross-hair
- Navigate around and then press the green + button to add a point
- Fill out the details, and then tap the tick in the top left.
- You’ve collected data!
Turning on Tracks
- Tap the menu in the top right to see your layers.
- Press and hold on your Tracks layer
- When the new menu appears, tap
Setup tracking - Here you can choose the interval for when a new point will be placed for your tracks
- As we’re inside and may not move much, turn on
Time requirementand set a shortMinimum timeof 15 seconds - Tap
Start tracking
- As we’re inside and may not move much, turn on
- If you have your phone’s GPS on, and you zoom in on yourself, your should see a tracking line appear.
- When the new menu appears, tap
Turning off Tracks
- Press and hold on your Tracks layer
- When the new menu appears, tap
Stop tracking
- When the new menu appears, tap
Exporting from QField to QGIS
To export your data back to QGIS, you can either copy the whole project folder in your phones storage back to your computer, or:
- Go to the menu
- Tap the folder with a cog on it
- Tap the three dots next to your data_collected.gpkg file
- Send it to your email
Instead of the email/transfer by cable option, you can instead create a QFieldCloud account. This is a more convenient approach, but you are limited to 100mb projects.
Stretch Goals
For Avenza Maps you just need to add a GeoPDF and you can collect data on it. It’s less functional than QField, but it’s a great fallback tool. Android Download iOS Download
There are alternatives out there, such as Input (MerginMaps).
Click on Show Layout Manager in the toolbar or use Project > Layout Manager. Create a new layout called “Avenza”. We can now see the Layout window.
Normally we would add many elements to our layout if we were exporting it for print such as the map, a legend, a scale bar, a north arrow…
In this case however, we are simply interested in our map. Let’s add the map to the canvas:
- Go to the Layout tab, scroll down to ‘Resize Layout to Content’, click ‘Resize layout’
- Before we export, let’s turn off any layers we aren’t using in QGIS to save space
- Click the Refresh View button up the top
- Now we are ready to export.
- Go to ‘Layout > Export as PDF…’ and save your map.
- The ‘PDF Export Options’ window will open
- Tick the ‘Create Geospatial PDF (GeoPDF)’ box
- Click ‘Save’
You can repeat this process with the DEM and Hillshade to export out another kind of map.
Now you simply need to export the pdf file(s) to your phone. You can email it, send it through the cloud, or transfer it using a cable.
When you first open Avenza Maps it will ask you to create an account, but you can import up to three maps without doing so, you can avoid creating an account by selecting the ‘x’ in the top right corner.
- Allow Avenza Maps to access your device location
- Select the orange ‘+’ in the bottom right and select ‘Download or import a map’
- Choose ‘From Storage Locations’ (if requested, give Avenza Maps the permissions to access your files)
- Do the same for the other map, if you created one.
- Once it has been imported, tap on the map.
- You can now move the map around with your finger, and pinching to zoom.
- Tapping the placemark icon in the bottom left will add a placemark in the middle of the crosshairs.
- Tapping the 3 dots in the bottom right will allow you to add GPS tracking and draw and measure distances.
We can use some fancy code to make the ID automated
- Double click on the new collected_data layer
- Click on the
Attributes Formtab - Click ID
- Under General untick
Editable- we don’t want to edit an automated field - Under Defaults click the
Expression Builderbutton- Paste in this code:
- Under General untick
- Click on the
concat(
"cat",
'-',
lpad(
to_string(
coalesce(
aggregate(
'DataCollection', -- layer name
'max', -- aggregate function
to_int(
regexp_substr("ID", '\\d+')
), -- numeric part of auto_id
"cat" = attribute(@parent, 'cat')
) + 1,
1
)
),
3,
'0'
)
)