I had a chance to generate UDP packets and send them to another host. This was needed for testing a specific problem that I had faced. The problem is that I had to see huge Recv-Q at "netstat -apln|grep -i 'syslog' prior to RHEL 5.4. Now question is why does it show such huge value? But why do we see this at first place? No, we didn't see it at first place. Actually one monitoring software was complaining that there were lot of failure d UDP packets in "netstat -s" output and it also complained that number of such packets were also huge in Recv-Q. So, I had to investigate in detail.
As you know Recv-Q = Number of bytes received from network as well as from local machine and that will be processed by syslog, so syslogd was not processing received UDP pkts.
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
udp 653212 0 0.0.0.0:514 0.0.0.0:* LISTEN syslog
As per my knowledge 0 0.0.0.0 = any.any.any.any address. And syslog collects messages from /dev/log. Also klogd sends kernel error/warning messages to syslogd. Syslogd filters the messages as per rules defined in /etc/syslog.conf and redirects them to corresponding log file. Big question is that if rule doesn't match then do the messages will be really going to log files(say /var/log/messages)? I think they will go if we accepts network messages. Pass "-r" to syslogd ie /etc/sysconfig/syslog.
---------> | /dev/log, klogd ----->| ------->syslogd ------>filter------->system log, @remote_server
---------> | ----->|
My concept about receiving the packets :
NIC -->NIC_DRIVER---->hrd_IRQ---->soft_IRQ---->PRE_ROUTING---->packet_forwarding---->libpcap(tcpdump)---->hook(INPUT)--->SKB--->CPU
I tried to send UDP packets locally and captured UDP by tcpdump. I have used server.c and rawudp.c two programs. So, server.c should be running to listen udp packets whereas rawudp.c will send udp packets. I took help from a guy to get these programs. I was trying to do using perl but there were lot of dependencies. So, I avoided it to use it. My intenstion was to verify whether Recv-Q was increasing by hitting UDP packets to server. But this does't increase. Note that I sent raw UDP packets. tcpdump data gave clear picture about source, destination, source and destination ports.
Here are the exact source codes and testing procedures :
------server.c---------
#include
#include
#include
#include
#include
#include
#define BUFLEN 512
#define NPACK 10
#define PORT 9930
void diep(char *s)
{
perror(s);
exit(1);
}
int main(void)
{
struct sockaddr_in si_me, si_other;
int s, i, slen=sizeof(si_other);
char buf[BUFLEN];
if ((s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP))==-1)
diep("socket");
memset((char *) &si_me, 0, sizeof(si_me));
si_me.sin_family = AF_INET;
si_me.sin_port = htons(PORT);
si_me.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(s, &si_me, sizeof(si_me))==-1)
diep("bind");
for (i=0; i
diep("recvfrom()");
printf("Received packet from %s:%d\nData: %s\n\n",
inet_ntoa(si_other.sin_addr), ntohs(si_other.sin_port), buf);
}
close(s);
return 0;
}
------server.c--------
------rawudp.c-------
/*
Copyright 2010 Gabriel Serme
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include
#define LEN 512
typedef unsigned short u16;
typedef unsigned long u32;
//2nd
unsigned short csum (unsigned short *buf, int nwords);
uint16_t udp_checksum(const struct iphdr *ip,
const struct udphdr *udp,
const uint16_t *buf);
int main(int argc, char * argv[])
{
# if __BYTE_ORDER == __LITTLE_ENDIAN //reverse per 8 bits
printf("little endian\n");
#else
printf("big endian\n");
#endif
if(argc != 5)
{
printf("- Usage %s
exit(1);
}else{
printf ("Args : \n"
"\tip source : %s:%s\n"
"\tip dest : %s:%s\n",
argv[1], argv[2], argv[3], argv[4]);
}
int PSOURCE = atoi (argv[2]);
int PDEST = atoi(argv[4]);
char * SOURCE = argv[1];
char * DEST = argv[3];
char * ANSWER = argv[5];
int s;
struct sockaddr_in daddr, saddr, answerip;
char packet[LEN];
/* point the iphdr to the beginning of the packet */
struct iphdr *ip = (struct iphdr *)packet;
struct udphdr *udp = (struct udphdr *)((void *) ip + sizeof(struct iphdr));
struct dnshdr *dns = (struct dnshdr *)((void *) udp + sizeof(struct udphdr));
if ((s = socket(AF_INET, SOCK_RAW, IPPROTO_RAW)) < 0) {
perror("error:");
exit(EXIT_FAILURE);
}
daddr.sin_family = AF_INET;
saddr.sin_family = AF_INET;
daddr.sin_port = htons(PDEST);
saddr.sin_port = htons(PSOURCE);
inet_pton(AF_INET, DEST, (struct in_addr *)&daddr.sin_addr.s_addr);
inet_pton(AF_INET, SOURCE, (struct in_addr *)&saddr.sin_addr.s_addr);
memset(daddr.sin_zero, 0, sizeof(daddr.sin_zero));
memset(saddr.sin_zero, 0, sizeof(saddr.sin_zero));
memset(udp, 0, sizeof(struct udphdr));
memset(ip, 0, sizeof(struct iphdr));
ip->ihl = 5; //header length
ip->version = 4;
ip->tos = 0x0;
ip->id = 0;
ip->frag_off = htons(0x4000); /* DF */
ip->ttl = 64; /* default value */
ip->protocol = 17; //IPPROTO_RAW; /* protocol at L4 */
ip->check = 0; /* not needed in iphdr */
ip->saddr = saddr.sin_addr.s_addr;
ip->daddr = daddr.sin_addr.s_addr;
udp->source = htons(PSOURCE);
udp->dest = htons (PDEST);
int sizedata = 100;
memset(((void *) udp) + sizeof(struct udphdr), 'A', sizedata);
int sizeudpdata = sizeof(struct udphdr) + sizedata;
ip->tot_len = htons(sizeudpdata + sizeof(struct iphdr)); /* 16 byte value */
udp->len = htons(sizeudpdata);
udp->check = udp_checksum(
ip,
udp,
udp);
printf ("Checksum : 0x%x\n", udp->check);
printf ("Sizes : \n\t[+] iphdr %d"
"\n\t[+] udphdr %d\n",
sizeof(struct iphdr), sizeof(struct udphdr));
printf ("Total size : %d\n", sizeudpdata);
int optval = 1;
if (setsockopt(s, IPPROTO_IP, IP_HDRINCL, &optval, sizeof(int))
< 0)
perror ("IP HDRINCL");
while(1) {
int sizepacket = sizeof(struct iphdr) + sizeudpdata;
if (sendto(s, (char *)packet, sizepacket, 0,
(struct sockaddr *)&daddr, (socklen_t)sizeof(daddr)) < 0)
perror("packet send error:");
else
printf("Sent packet\n");
sleep(2);
}
exit(EXIT_SUCCESS);
}
//http://www.linuxquestions.org/questions/linux-networking-3/udp-checksum-algorithm-845618/
//modified by Gabriel Serme
struct pseudo_hdr {
u_int32_t source;
u_int32_t dest;
u_int8_t zero; //reserved, check http://www.rhyshaden.com/udp.htm
u_int8_t protocol;
u_int16_t udp_length;
};
uint16_t udp_checksum(const struct iphdr *ip,
const struct udphdr *udp,
const uint16_t *buf)
{
//take in account padding if necessary
int calculated_length = ntohs(udp->len)%2 == 0 ? ntohs(udp->len) : ntohs(udp->len) + 1;
struct pseudo_hdr ps_hdr = {0};
bzero (&ps_hdr, sizeof(struct pseudo_hdr));
uint8_t data[sizeof(struct pseudo_hdr) + calculated_length];
bzero (data, sizeof(struct pseudo_hdr) + calculated_length );
ps_hdr.source = ip->saddr;
ps_hdr.dest = ip->daddr;
ps_hdr.protocol = IPPROTO_UDP; //17
ps_hdr.udp_length = udp->len;
memcpy(data, &ps_hdr, sizeof(struct pseudo_hdr));
memcpy(data + sizeof(struct pseudo_hdr), buf, ntohs(udp->len) ); //the remaining bytes are set to 0
return csum((uint16_t *)data, sizeof(data)/2);
}
/* Not my code */
unsigned short csum (unsigned short *buf, int nwords)
{
unsigned long sum;
for (sum = 0; nwords > 0; nwords--)
sum += *buf++;
sum = (sum >> 16) + (sum & 0xffff);
sum += (sum >> 16);
return ~sum;
}
--------rawudp.c--------
Compile programs :
$gcc -o server server.c
$gcc -o rawudp rawudp.c
You'll see server and rawudp files.
--------------------------------
Testing :
./server //server is ready to listen UDP which will be sent from rawudp prgram.
$tcpdump -i lo -X -vv 'port 9930' -w captured.pcap //I am sniffing it on another terminal.(Note that server port is 9930)
./rawudp 127.0.0.1 1111 127.0.0.1 9930 //sending raw packets to localhost on another terminal(destination is 127.0.0.1 and its port is 9930. source port is 1111 )
src src_port dst_h dst_p
$wireshark captured.pcap //Viewing captured UDP packets
------------------------------
Conclusion :
a) Firewall was enabled : $iptables -A INPUT -s 127.0.0.1 -p udp -j DROP ,
So, ./sever did't listen any udp packets. //Test was done
Ou put :
./server
Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA //16bytes
Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
These messages will be stopped once I again execute "iptables -A INPUT -s 127.0.0.1 -p udp -j DROP". But I saw that number of packets capturing in tcpdump was increasing. And rawudp was continuosly sending udp packets.
Result :
[root@vm ~]# tcpdump -i lo -X -vv 'port 9930' -w gaby.dump3_blokupd
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
Got 37
[root@vm122 myUDP]# ./rawudp 127.0.0.1 1112 127.0.0.1 9930
little endian
Args :
ip source : 127.0.0.1:1112
ip dest : 127.0.0.1:9930
Checksum : 0x3317
Sizes :
[+] iphdr 20
[+] udphdr 8
Total size : 108
Sent packet
Sent packet
Sent packet
Sent packet
. . . .
---------------------
Why Recv-Q is so high :
1. DNS server was sending lot of UDP(LLMNR) packets.
2. Local machine was sending lot of UDP(LLC) packets.
3. Some machines which are in same subnet were sending UDP to other machines(Don't know why libpcap captured them. Did they route after passing through tcpdump? I still have doubt)
4. Increase size of rmem kernel memory.
0 comments:
Post a Comment