====== Cloud data storage ======
//Required knowledge: 
[HW] [[et:hardware:homelab:controller]], [HW] [[et:hardware:homelab:digi]], \\
[AVR] [[et:avr:adc]], 
[LIB] [[et:software:homelab:library:module:sensor]], [LIB] [[et:software:homelab:library:module:ethernet]]//

===== Theory =====

Cloud data storage means sending data over a wired or wireless network to an external server. The external server can be local, a file server in your own corner, or a well-known cloud service provider. Cloud services are offered by many large IT companies and by companies providing Internet of Things services. Some are free and open-source servers, others are closed, paid services aimed at companies. To store microcontroller data in the cloud, you need to know the service specifics, communication protocol, and secure data transfer parameters. Data storage usually takes place over a standard Internet interface (Ethernet) using the TCP/IP protocol.

The main advantage of cloud storage is secure data preservation and broad possibilities for later data processing. Service providers often offer powerful tools for visualization, analysis, and presentation. With such tools it is convenient to create web pages that show data as charts, even though the data is collected by a microcontroller. Anyone can also set up their own cloud server and start storing data there. It is possible to use some open-source platforms or build a simple application yourself. The do-it-yourself approach requires additional resources in server hardware, network connections, and your own skills and time.
{{ :et:examples:storage:temp_logi_veebileht.png?500|Web page for displaying and visualizing data }}
===== Practice =====
To store data in the cloud, you first need to choose a suitable cloud service or set up your own cloud server. It is easiest to use existing large-company services. At the same time it is quite simple to build the required functionality yourself, especially if you already have a server and prior experience with web pages and database systems.
Since the list of cloud service providers is constantly changing and their pricing and functionality also change, the following example is a minimalist solution using an existing web server with Apache2, PHP5, and a MySQL database engine. Based on this example you can always make the solution more complex and complete; there are many guides on how to build database-backed web pages and visualize data.

The example application uses a simple temperature sensor that stores a temperature reading at a certain interval and sends it to the cloud server. Communication uses a wired Ethernet interface. The solution consists of two parts: microcontroller software and web server software.

First, the web and database server are configured by creating an empty database and a web page that receives data from the controller and shows it to the user in a table and as a graph. The graph is generated using the Google Visualization API. The database can be created using phpMyAdmin or the mysql client program. In this example the database is named //pilvedemo// and the table is named //monitor//.

<code sql>
create database pilvedemo;
</code>

<code sql>
CREATE TABLE monitor (
`id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY COMMENT 'unique ID',
`event` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Event Date and Time',
`sensor` VARCHAR( 30 ) NOT NULL COMMENT 'Sensor ID',
`value` VARCHAR( 10 ) NOT NULL COMMENT 'Measured Sensor Value',
INDEX ( `event` , `sensor` )
) ENGINE = InnoDB;
</code>

After the database is created, you need to create a user and grant permissions to read and write the database data. In the following example the database user is //monitor// and the password is //demo//.
Next, you create PHP code in a directory accessible by the web server that displays the database data in a table and also stores the data sent by the controller into the newly created database.

<code php>
<?php 
// Save temperature sensor readings to the database and display them on the web
$dbh = mysql_pconnect("localhost" , "user", "password");
$selected = mysql_select_db("pilvedemo",$dbh);

if (($_GET["s"])&&($_GET["v"])){
   $SQL = "INSERT INTO monitor (sensor,value) 
            VALUES ('".$_GET["s"]."', '".$_GET["v"]."')";    
    mysql_query($SQL);
}

$result = mysql_query("SELECT date(event) aeg,sensor,value 
                        FROM monitor ORDER BY id ASC");

while ($row = mysql_fetch_array($result)) {
   // Build the table
   $tabel .= '<tr><td>'.$row["aeg"].'</td>'."\n";
   $tabel .= '<td>'.$row["sensor"].'</td>'."\n";
   $tabel .= '<td>'.$row["value"].'</td></tr>'."\n"; 
   // Build data for the chart
   $date = date("Y,m,d", strtotime($row["aeg"]));
   $mod_date = substr_replace($date, intval(substr($date, 5,2))-1, 5,2) ;
   $chart.= "\t\t[new Date(" . $mod_date . ")," . $row["value"] . "],\n";
}
$chart.= substr_replace($chart ,"",-2); // Remove trailing comma from last element
?>

</code>
<code html>
<!DOCTYPE html>
<html>
<head>
  <title>Cloud storage demo server</title>
  <meta http-equiv="Content-Type" content="text/html;charset=UTF-8"> 
  <style type="text/css">
   table {font-family: verdana,arial,sans-serif; font-size:11px; color:#333333; 
                                                               border-width:1px;}
   th {background:#b5cfd2; border-width:1px; padding:8px; border-style:solid; 
                                                          border-color: #999999;}
   td {border-width:1px; padding:8px; border-style:solid; border-color: #999999;}
  </style>

   <script type="text/javascript" 
    src="https://www.google.com/jsapi?autoload={'modules':[{'name':'visualization',
                                      'version':'1','packages':['corechart']}]}">
   </script>
    
   <script type="text/javascript">	
	  google.load('visualization', '1', {packages: ['corechart']});
	  google.setOnLoadCallback(drawChart);

	  function drawChart() {
	    var data = new google.visualization.DataTable();
	    data.addColumn('date', 'X');
	    data.addColumn('number', 'Outside temp.');
	    data.addRows([<?php echo "\n".$chart?>]);
	    var options = {width:500,height:300,hAxis:{title:'Time',format:'dd.MM'},
	    	    	   vAxis:{title:'Temp. C'}};
	    var chart = new google.visualization.LineChart(
	    document.getElementById('temp_chart'));
	    chart.draw(data, options);
	  }
	  </script>    
</head>

<body>
    <h1 style="font-family:verdana,arial,sans-serif;font-size:26px;color:#333333;">
    Temperature Log</h1>
    <table><tr><td>
    <div id="temp_chart"></div>
    </td>
    <td>
    <table>
      <tr><th>Time</th><th>Sensor</th><th>Reading</th></tr>
	<?php echo $tabel?>
    </table>
    </td>
    </tr>
    </table>
    Robolabor.ee
</body>
</html>

</code>
<pagebreak>

The microcontroller code must be compatible with the server code, especially in terms of data format and URL. In this example, the data display URL is http://robolabor.ee/pilvedemo/index.php and the data upload URL is http://robolabor.ee/pilvedemo/index.php?s=<id>&v=<reading>
parameter s is the sensor ID and parameter v is the sensor reading. The timestamp is added automatically for each entry based on when the data arrives.

The following code is a HomeLab example program that matches the server and database configuration described above. For HomeLab III you need a separate Ethernet module, which is included in the Robotic HomeLab add-on kit. HomeLab III connection schematic:
^ Signal ^ Pin ^
| SI      | PF3  |
| SO      | PF2  |
| SCK     | PF1  |
| CS      | PF0  |
| VCC     | +3.3V |
| GND     | GND  |

<code c>
// HomeLab temperature reading cloud storage example program
// The reading is taken once every 5 seconds and stored in the cloud server database
#include <avr/io.h>
#include <avr/interrupt.h>
#include <util/delay.h>
#include <stdlib.h>
#include <adc.h>
#include <module/sensors.h>
#include <module/ethernet.h>

// Define server address
#ifdef WEBSERVER_VHOST
    #undef WEBSERVER_VHOST
#endif
#define WEBSERVER_VHOST "robolabor.ee"

// HomeLab II
//#define ADC_CHANNEL 2
// HomeLab III
#define ADC_CHANNEL 14

// Packet buffer size
#define BUFFER_SIZE 650
uint8_t buf[BUFFER_SIZE+1];

// Main program
int main(void)
{
    uint16_t time = 0;

    // Start Ethernet controller, obtain IP
    ethernet_init_client();
    ethernet_getDNSIP();
    ethernet_get_server();

    // Configure ADC
    adc_init(ADC_REF_AVCC, ADC_PRESCALE_8);

    // Infinite loop
    while (1)
    {
        uint16_t dat_p,plen;

        // Keep Ethernet link alive
        plen=enc28j60PacketReceive(BUFFER_SIZE, buf);
        buf[BUFFER_SIZE]='\0';
        dat_p=packetloop_arp_icmp_tcp(buf,plen);

        if(plen == 0)
        {
            // Send new data to the server every 5 seconds
            if(time >= 5000)
            {
                itoa(thermistor_calculate_celsius(adc_get_average_value(
                                                 ADC_CHANNEL, 4)),urlvarstr,10);
                                                 
                client_browse_url(PSTR("/pilvedemo/?s=1&v="),urlvarstr,
                      PSTR(WEBSERVER_VHOST),&browserresult_callback,
                      otherside_www_ip,otherside_www_gwmac);
                      
                time = 0;
            }
            _delay_ms(1);
            time++;
            continue;
        }
        
        if(dat_p==0)   udp_client_check_for_dns_answer(buf,plen); // plen!=0
    }
}
</code>