Which cpu to save a lot of data and how to do it

Hi all,

I have been assigned a job that includes to save and export a lot of data.

My plan is to take some help from Siemens for this application but first i would like to discuss this here on the forum to get as much feedback as possible.

The machine is a test rig that run up to 3million cycles, each rig cycle the PLC  should save some data, lets say 10 analog values (Real).

When the rig has finished its test cycle the data should be exported, and here comes our weak point on the company, we don't have much experience to save that big amount of data. In the past we have done some machines that save data to excel but not near the amount of data this applications requires. We have been given free hands from the client to find the best solution to export the data in any form, so all thouhgts are welcome .

About the hardware we are thinking either

A S7-1500F in some form with a Wincc runtime advanced as HMI

OR

ET 200SP Open controller, CPU 1515SP PC F + HMI 512PT

So for my questions:

1) The excel choice feels a little bit scary, store 3 million rows of data.  Maybe we should save the data in a database?

2) What do you think about the hardware for this purpose? The ET200SP feels like a better choice if a database is to prefer to save all data.

4) Is there any Siemens solutions software that can take care of this kind of data and make some kind of reports or present it in some way?

If we choose a database solution we are going to hire a consultant to do the data logging, but before we need to get some knowledge about this so we could make the right hardware choice.

Looking forward for every thought about this

Thanks in advanced

Greeting David R

Hi David,

What kind of data acquisition rate are you anticipating?

I concur that excel at 3 million rows is excessive. I currently have a system where we have several hundred S7 PLCs that are remotely polled via a VPN connection. A central server polls the PLCs once a minute to retrieve a set of data in the order of 2kBytes and then processes it into alarm messages by decoding a block of discreet bits and other, analog, data is stored into a MySQL database.

Once the data is in the database you are 'off to the races' so to speak! We have remote access to the data via the usual web server techniques. My system is custom built but you can purchase similar solutions from other companies out there. That may be more expensive, but if you are not trying to grow into 1000's of PLCs; it is probably a more economic proposition to use a third party solution such as MB Connect.

I'd be interested to know what solutions you come up with!

Simon

Hi Simon, 

One test rig cycle takes 5-10 seconds.

Thanks for sharing your knowledge and experience in this subject, i will read up on the  MB Connect.

Im gonna share my work how it evolves and how the thoughts go.

Simon Gibson

..., data is stored into a MySQL database.

...

THAT is the deal, I think so!

Hi David, 

roughly speaking, you have 10 reals (10*4 bytes), 3M samples that gives about 240MBytes of raw data.. this is a quite large amount   perhaph you need a reliable logging system.

The way you want to use data for post processing shoud be consideret to plan the way data will be collected and stored.

For collecting data I would suggest by profinet and S7 protocol or modbus TCP protocol... There are different tools that are capable of read and store data for you at regular interval.

For example give a try to this.

More professional and generic (see reusable) software you sure can get using Modbus client.

For Communication PC-PLC I'm using a S7 third party library and develop custom application using .NET Visual Studio and to store data i use SQLServer or SQLServerLite.

It depends on the sight of your application.. maybe it should be a worse to develop a new app or use a simple testing tool?

Please let us up to date on your advances.

Bast recards,

Luca.

Thanks and ratings if helping is appreciated.

Hi all,

Thanks for all your input.

I discussed with the client today, and they had a idea that we could do daily reports to excel, but this feels for me like a bad idea, the test rig take several months for one test cycle so this would be a lot of excel files, and i could imagine that in the future the client wants to look at the data more than daily.

I looked at your link lucavanzin about excel logging, and after some googling i find a german company that looks interesting www.plcreports.com/, i will contact them for more information about there product, they used as you wrote Profinet interface to read data. They have a license that is unlimited , feels like this is for me, maybe not for the wallet

The plcreports company feels like an alternative if we choose to do the data logging "in house", but first we need to know the limitations and "how to do" about the software.

The SQL alternative feels more "open" for in the future do what ever we want with the data, the key element feels like to get a good structure from the beginning or?

Another Way you could use is using internal DataLog of PLC.

If you use 1500 CPU, it is capable to save raw data in binary mode, so you could keep data space at minimum.. this imply that for data reading you use a sort of decoder... I don't know if there are some Siemens tool to do that. 

Otherwise, creating a clear text datalog, once a day, you will take a lot more space (say... 10 ascii char for every value, 10 values, 10 seconds save period... is about 864k each file) 

If you get a sample every 10 sec, to totalize 3Millions samples it take 347 days 24hs/day

So you will have 347 files and the total space required in program memory will be about 300MB

You could get a SIMATIC Memory card 512MB and you could store an entire data set in text format with index and timestamp.

Enabling Web server, you can Browse and download Log files as you like, also the current running log. You can delete then file from Memory card, saving space. 

If you know that you will access the file every X time... you could also program the Log as overwriting, so you will keep one small file and get only the new data when you get that.

To automate recover data, you will need then a tool to get files from CPU and import as needed to further elaboration.

Hi all,

Some update about the project.

Fritz, thanks for the tip about Iba software, i will Contact them for further information.

I haved contacted PLC Reports and it looks intresting, you pay for the quantity of varibles you want to logg, so in this case we will pay a license for 10 varibles. The software has watchdog function and able to write to PLC:s tags. It will be intresting to compare this to the Iba software.

We have also contacted Omron, they have in their PLC:s direct acess to SQL database via the built in Ethernet/IP port, so if we go the SQL way maybe their solution is a alternative, but then we still need a .NET Consultant.

It all depends on how the client want thier data structured, so it take some more meetings to clear this out.

About the PLC hardware im thinking the:

 SIMATIC ET 200SP OPEN CONTROLLER, CPU 1515SP PC F + HMI 2048PT

https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7677-2FA41-0FM0

It feels like it has all the ingredients for the Projects, with safety and even if will be a SQL solution or Excel sloution this PLC would do the job. Compared to the Omron the Siemens was cheaper and you get "all in one box".