Good morning,

I have a quite large ethernet network. In the nework I have more or less 30 stations, each one with its own S7/1500 PLC and profinet slaves.

All the PLC are also I-Device slave of the "master" PLC.

In this big network I have 300 or more nodes, many of them are profinet nodes, others are generic TCP/IP nodes (mostly PC).

Sometimes it happens that specific broadcast messages (Netbios dns, Dropbox sync protocol) trigger a lot of ARP requests, made by PC nodes (laser markers, servers, SCARA controller, etc.)

When the network is flooded with those ARP requests, some PROFINET nodes raise an error and stop communicating.

Wireshark report the following error RTA_ERR_ABORT. After some research, it seems that the profinet interface goes out of error.

So my question are:

- am I wrong? Anyone had a similar problem?

- how could I redesign the network to prevent spreading of broadcast messages? Do I need routers/gateway on each station?

 Hi currently im having same problem with SEW devides on my profinet network... The devices gos down on profinet 3-4x in hour with same error ... I found it using wireshark with what devices do you have problem ?  I have same error RTA_ERR_ABORT

rbresca

Good morning,

I have a quite large ethernet network. In the nework I have more or less 30 stations, each one with its own S7/1500 PLC and profinet slaves.

All the PLC are also I-Device slave of the "master" PLC.

In this big network I have 300 or more nodes, many of them are profinet nodes, others are generic TCP/IP nodes (mostly PC).

Sometimes it happens that specific broadcast messages (Netbios dns, Dropbox sync protocol) trigger a lot of ARP requests, made by PC nodes (laser markers, servers, SCARA controller, etc.)

When the network is flooded with those ARP requests, some PROFINET nodes raise an error and stop communicating.

Wireshark report the following error RTA_ERR_ABORT. After some research, it seems that the profinet interface goes out of error.

So my question are:

- am I wrong? Anyone had a similar problem?

- how could I redesign the network to prevent spreading of broadcast messages? Do I need routers/gateway on each station?

Staubli Scara controller RC8 mounting Applicom Profinet card, are the nodes that suffer the most this condition.

But also some Kistler Press Controller.

I think it depends of the profinet low-level chip. Maybe those one have the smallest buffer.

I wonder why Scara controller (running wx-works) send a lot of ARP request after a Drop box discovery protocol request.

Those ARP request are subnetwork requests (they do not ask the address of a single IP, they ask the address ending with 255, thus asking the mac address of all the nodes of the subnet).

Also, I don't understand how those ARP requests could affects profinet nodes, forcing them to ABORT the communication.

 I deeply analzyed my problem on network.. And I see forexample that all messages was send to slave so there is no need to close connection.. (frame analyzing on layer2)

For now only one think works to set the watchdog time on sleave to high value.. On simatic it is number of cycles withou data. with forexample 2ms cycle time and valuee 70 it is 140ms watchdog time..  Just as example your cycle time can be different.. BE careful do not change czcle or update time just watchdog!

Noramlly this is 100% error on the suplier side because all PN frames was send and rcvd from device interface.. so no reason to cut connection from slave... because of frame monitoring watchdog...

Try to use this woreraround for now but be carefull this is watchdog taht delay the detection time of profinnet lost.. In high speed aplication it can be dangerous.. 

You can reduse amount of messages using managed profinet switches but i believe thats not the problem because profinet is builded for bigger load e specialy in timing IRT operations..

Do not build long chain architectures every switch port on device add some delay to comunication..

Staubli Scara controller RC8 mounting Applicom Profinet card, are the nodes that suffer the most this condition.

But also some Kistler Press Controller.

I think it depends of the profinet low-level chip. Maybe those one have the smallest buffer.

I wonder why Scara controller (running wx-works) send a lot of ARP request after a Drop box discovery protocol request.

Those ARP request are subnetwork requests (they do not ask the address of a single IP, they ask the address ending with 255, thus asking the mac address of all the nodes of the subnet).

Also, I don't understand how those ARP requests could affects profinet nodes, forcing them to ABORT the communication.

I don't think I have problems on profinet side.

Analyzing the packets with wireshark, I can see that, despite the high load caused by ARP packets, profinet communication is regular (every 4 ms, as configured).

Nevertheless, the slave gives up with RTA_ABORT_ERROR. It is not a watchdog error.

RTA_ERR_ABORT is default abort class on PN

You must check Error code 2 on profinet.. If it is 5 thats mean Watchdg time...

0x02 instance closed (RTA_ERR_ABORT)

0x03 AR out of memory (RTA_ERR_ABORT)

0x04 AR add provider or consumer failed (RTA_ERR_ABORT)

0x05 AR consumer DHT / WDT expired (RTA_ERR_ABORT)

0x06 AR cmi timeout (RTA_ERR_ABORT)

0x07 AR alarm-open failed (RTA_ERR_ABORT)

0x08 AR alarm-send.cnf(-) (RTA_ERR_ABORT)

0x09 AR alarm-ack- send.cnf(-) (RTA_ERR_ABORT)

0x0A AR alarm data too long (RTA_ERR_ABORT)

....... And more.. All are RTA_ERR_ABORT but error code 2 says what is reason...

 Can you post your wireshark log picture with your wireshark log...

rbresca

it is 0x02: "instance closed"

Hello,

in the attachment you will find three snippets.

1: normal communication, there are almost only profinet packets

2: this is the instant when the ARP storm started. Probably caused by DropBox discovery request

3: death of a profinet node! Note that previous packets are regular. Every 4 millisecond the is a request/answer, even if the answer is somewhat delayed.

Then, suddenly, the RTA_ERROR. I forgot to highlight that row in the image, I'll put it below as text

Frame 215391: 56 bytes on wire (448 bits), 56 bytes captured (448 bits) on interface 0
Ethernet II, Src: Applicom_10:c2:23 (00:a0:91:10:c2:23), Dst: Siemens_26:fd:ce (ac:64:17:26:fd:ce)
PROFINET acyclic Real-Time, Alarm Low, ID:0xfe01, Len:  40
FrameID: 0xfe01 (Real-Time: Acyclic PN-IO Alarm low priority)
PROFINET IO Alarm
AlarmDstEndpoint: 0x0001
AlarmSrcEndpoint: 0x0000
PDUType, Type: ERR-RTA-PDU, Version: 1
AddFlags, Window Size: 1, Tack: 0
SendSeqNum: 0xffff
AckSeqNum: 0xfffe
VarPartLen: 0x0004
Status: Error: "RTA error", "PNIO", "RTA_ERR_CLS_PROTOCOL", "Instance closed (RTA_ERR_ABORT)"
ErrorCode: RTA error (0xcf)
ErrorDecode: PNIO (0x81)
ErrorCode1: RTA_ERR_CLS_PROTOCOL (253)
ErrorCode2: Instance closed (RTA_ERR_ABORT) (2)

No one has some hints?

Yesterday I have found another source that is capable to trigger a lot of ARP requests.

We have a software which uses HASP keys. Hasp sends UDP multicast packets to find licence managers in the network.

Those packets trigger the "ASP storm" which causes some profinet nodes to give up and sens the RTA_ERROR.

For the moment I have bypassed this issue adding a rule in programmers laptop to block outbound upd packets toward port 1947.

But I will have no control when we will are in the client plant.