Efficient working with Databases in LabVIEW - Sam Sharp (MediaMongrels Ltd) - GDevCon#2
•Transferir como PPTX, PDF•
2 gostaram•2,607 visualizações
Sam Sharp's presentation from GDevCon#2 on Efficient Working with Databases in LabVIEW. This presentation discusses some best practice hints & tips for working with databases in LabVIEW and uses Yii's ActiveRecord implementation as an example of how we can work more efficiently with databases in LabVIEW.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Efficient working with Databases in LabVIEW - Sam Sharp (MediaMongrels Ltd) - GDevCon#2
Semelhante a Efficient working with Databases in LabVIEW - Sam Sharp (MediaMongrels Ltd) - GDevCon#2 (20)
Último
Último (20)
Efficient working with Databases in LabVIEW - Sam Sharp (MediaMongrels Ltd) - GDevCon#2
- 1. Efficient Working with Databases Sam Sharp MediaMongrels Ltd sam@mediamongrels.com
- 2. WebSockets API • New version (V2.X) coming soon! – New OO-based API – Available on NI Tools Network Client: Server: WSS Support?!
- 3. Introduction • Considerable amount of LabVIEW & Database exposure: – PHP/MySQL Web Development (4-5 years) – UK Fuel Cell Company • Test Station Monitoring System (~100 Test Stations) • 60s Update Resolution (Test/Asset Status, PC Health) – Swiss Light Source Company • Design & Rollout of Production Test Database (10-15 tables) • TestStand Results Processing Plugin & LabVIEW API • Reporting & Analysis Tool in LabVIEW • Legacy Data Import • Writing VIs to talk to databases in LabVIEW is tedious.
- 4. Databases Overview • Database Advantages: – Store large amounts of information – Sharing of information • Concurrent multi-user access – Quick access to information – Eliminate duplication of data – Information Security – Backup/Restore • Common DBMS: – Relational • Linked Tables (Rows & Columns) • Examples: MySQL, MS-SQL, Oracle, PostgreSQL, SQLite – NoSQL • Unstructured/Semi-Structured Data • Examples: MongoDB, Redis, Cassandra • LabVIEW <-> Database – Database Connectivity Toolkit – 3rd Party Toolkits (SQLite, GDatabase)
- 5. Database Toolkit Basics ODBC Drivers MySQL: MySQL ODBC X.X ANSI Driver MS-SQL: SQL Native Client LV 32-bit
- 6. Best Practice Hints & Tips • Security – Use parameter binding for user input (SQL injection) – Passwords in VIs? – Limit application access to databases by user roles • Performance – Do not repeatedly open/close the connection – Database design… …but what if the username & password is: “ OR “1”=“1 SELECT * FROM operator WHERE username = ““ OR “1”=“1” AND password = ““ OR “1”=“1” Any username/password will grant access! Looks harmless (and I have seen this done in many places!) https://xkcd.com/327/
- 7. Database Design • Database design is important – Poorly designed database issues: • Poor quality data • Poor performance – Especially when volume of data increases • Redundant data • Poor data integrity • Difficult to extend/improve • Normalisation Principles – 1NF, 2NF, 3NF etc. – Guide good relational database design • Each relation adds complexity – sometimes it is sensible to break to rules. – e.g. execution_id in step_measurement – Storing JSON configuration (for audit purposes only)
- 8. Question: How long would it take to write code to Create, Read, Update & Delete rows for these five tables?
- 10. In ~90 seconds* we have: • Configured a database connection • Generated model classes for 5 tables – One model class per table – Columns as model properties – Basic validation (e.g. required fields) • Created a user interface to create, view, update and delete rows from our Operator table … without writing a single SQL query or line of PHP code …leaving more time to develop the ‘meat’** of the application …the joy of frameworks! A prize for anyone that can rewrite as a Haiku! *sequences have been shortened (no lawsuits please!) **or your favourite soy-based substitute
- 11. Yii PHP Framework • Popular framework for developing web-based applications – PHP & MySQL – Based on Model-View-Controller (MVC) architecture • Model classes represent object/table data (e.g. customer, order) • Views display data to the user • Controllers implement the ‘business logic’ by interacting with the models and preparing the views – Contains tools for auto-generating code from a database schema
- 12. What if we could do some of this in LabVIEW?
- 13. ActiveRecord • ActiveRecord implements the Models in Yii’s MVC – ActiveRecord objects contain persistent data and behaviours operating on that data • Class = Table • Object = Row • Properties = Columns • Methods = Behaviours (e.g. create, read, update, delete, business logic) • Purpose: – Representing models and their data – Representing relations between models (e.g. foreign keys) – Validating models – Performing database operations +Insert() +Update() +Delete() +GetByID() +CheckPermission() -id -username -full_name -level Operator
- 15. In ~150 seconds* we have: • Configured a MySQL connection (ish) • Generated model classes for 5 2 tables – One model class per table – Columns as model properties – Basic validation (e.g. required fields) • Created a user interface VI to create, view, update and delete rows from our Operator Test Execution table … without writing a single SQL query or much LabVIEW code …leaving more time to develop the ‘meat’** of the application …the joy of frameworks! *sequences have been shortened (no lawsuits please!) **or your favourite soy-based substitute
- 16. Current Features • ActiveRecord base class – Create, Read, Update & Delete of Objects – Automatic creation of model classes using Scripting • Supporting Libraries / Tools – Database Connection Class • Database Connectivity Toolkit – Query Command Library • Builds SQL queries from VIs – Integrated into LV Tools folder
- 17. Future Functionality • Query Options – FindAll, FindByAttributes etc. • Model Validation • Foreign Key Relations – Test_Execution.Test_Steps • User Interface Binding?
Notas do Editor
- Good afternoon, I hope everyone enjoyed their lunch and feel free to use the next half an hour for a nap if you’re feeling a little sleepy. My name is Sam Sharp, I run my own little NI Alliance Partnership, MediaMongrels Ltd, I recently became a LabVIEW Champion (Hooray!) and I am one of the founding members of GDevCon so feel free to come and talk to me if you have any feedback. Today I am going to be talking about communicating with databases in LabVIEW…but first…
- Another quick announcement – a new version of my LabVIEW WebSockets API is going to be released soon on the NI Tools Network. It’s the first major update to the library since its release that simplifies the API, fixes some bugs, adds some features and also allows for Secure WebSockets Support in the future. A version for LabVIEW NXG to use with the NXG Web Module will also be coming soon.
- OK, so onto my presentation. Most of my career dating back even to school has involved some form of work with databases, whether as a Web Developer writing applications in PHP/MySQL or writing LabVIEW applications that interface with databases. One thing I find when working with databases in LabVIEW is that it is TEDIOUS. Writing VIs to talk to a database with anything more than a few tables takes ages and much of it is repeated work (insert into X, insert into Y etc.). It is a BORING and THANKLESS TASK.
- Before I get to the real point of this presentation, I just wanted to give a bit of an introduction to databases over the next few slides. If you need any of the things on the left-hand side of the slide, you should probably put down Excel or your file-based record keeping system and consider using a database. Databases come in different shapes and sizes. Relational: Structured tables with rows and columns defined in a schema. Write queries (SQL) to read, update, delete and create data. Each row has a unique key. NoSQL: Allows unstructured and semi-structured data to be stored/manipulated. Data is stored as ‘documents’ (e.g. JSON in MongoDB), Key-Value pairs (Redis). When it comes to talking to databases in LabVIEW – there’s the built-in database connectivity toolkit and other 3rd party toolkits – such as James Powells excellent SQLite library which he will be talking about after me.
- Since the DCT is included in LabVIEW, here’s what a basic database operation in LabVIEW might look like…we open a connection (yes, you can wire different data types here), select some data, convert it to a LabVIEW data type and then close the connection. Depending on the database type you’re talking to, you might need to install some additional drivers and configure your database connection.
- Here are a few handy hints & tips to help you build applications that talk to databases in a robust and secure way.
- Having a poorly designed database can have a detrimental effect on your business. It can result in poor performance, data loss etc. Fortunately, the normalisation principles are there to help you. The normalisation principles (E.g. 1st normal form, 2nd normal form etc.) are there to help guide the design of your relational database. They are a tool to help you, but it is up to you whether to use them. As each added relation between tables adds complexity to your database design, sometimes it is sensible to break the rules. Here we have added the execution_id to step_measurement to give us a faster way to look up all measurements for a particular test execution.
- Ok, now you’re clued up on what a database is, how to design it and then talk to it from LabVIEW, let’s see if we can make our lives easier and remove some of the tedium involved and spend more time writing the interesting parts of our applications? Given our database design from the previous slide, how long it would take to write LabVIEW VIs for these five tables? Anyone want to shout out some guesses? A day? Half a day? A few hours?
- What if I said it could be done in 90 seconds? You’ll immediately notice that I’m cheating and this isn’t LabVIEW but here goes… First I configure a database connection to talk to my test database. I can then go to a special link, click the Model Generator and enter an asterix to generate all models, you’ll then see that it has populated a list of all of my database tables. I can preview the file to see the columns in the table and then generate my model files. I can then go back and go into the CRUD generator, configure my Operator controller, generate the files and then go to the URL for my Operator Controller. This has now added a user interface where I can add a new operator, view operators, update an operator and delete them.
- This was all done in the Yii Framework, a PHP framework for writing web-based applications. It is based on the Model-View-Controller architecture where Model… View… Controller… It also happens to contain tools for auto-generating the models, views and controllers automatically from a database.
- So – what if we could do some of this in LabVIEW?
- In the Yii Framework, database models are represented using ActiveRecord classes. AR objects contain persistent data (e.g. stored in a database) and have behaviours acting on that data. An AR class represents a table, an instance/object represents a single row in the table and properties and methods represent table columns and behaviours respectively. The purpose of ActiveRecord is to represent models and their data, relations between models and to perform validation and database operations. On the right you can see the progression from database table, to AR class design, to LabVIEW class to application code using the Operator ActiveRecord class.
- So here we are in LabVIEW… First I’m going to open up the ActiveRecord Model Generator…connect to my database and select the Operator table. I want the ‘Level’ field to be an enum so I’ll change that here and then generate the model class. I’m also going to do the same for my Test Execution table. The database data types don’t always match our LabVIEW types so I’m just going to fix a couple of them here before we generate the model. Now in our project folder we have a couple of folders containing our classes which we can then add to our project. The first thing to do is to populate the enum type definitions that have been automatically generated with the values that we want for our Operator Levels and Test Execution status fields. We can now start writing our application – for this demo I’m just going to create a new row in our Test Execution table. So I drop down a new test execution object, set the properties (which are our database columns), add a ‘Save’ VI and then read out the ID of the newly inserted database row just to verify that it worked.