Torrenting...best: practices/FW version/tweaks

[jail4ceos2]

Hi

Recently I've noticed my download speed degrades over several hours. I torrent about 15GB for about 10ish torrents per day on average, on a 60MB/16MB PPPoE ADSL connection (torrents done on my HTPC connected via ethernet). At first I thought the degradation was caused by my ISP throttling me as uTorrent's "speed" tab would show the traffic starting as varying widely up to my limit and down to half of it, but then my traffic would eventually become basically a flat line at less than 5% of my max speed. I would reboot the ISP modem and router, and I'd be back to my max. One day I just rebooted the RT-N66U, and I was back to max from 5%. While this is nothing but anecdotal, I feel the router is perhaps not configured to survive the abuse that torrenting at this level forces on it. I do have QoS on, but not HW acceleration. I have between 5-10 other devices on wireless (not torrenting, just general surfing), and they can still do everything flawlessly. I do generally shut down networking before bed, so at most my issues seem to occur over a 2-3 hour period once the daily torrent load starts.

What do folks, including you RMerlin, suggest I should do to gain more consistent performance? Are there scripts I can use to manage the memory better if the huge amount of torrent connections is the issue, run the torrent client on the N66U..or...I dunno, but I'll bet the bright minds here know how to tweak the settings to optimize for consistent max torrenting, while maintaining overall LAN performance and stability. I hope this thread will become a repository for good ideas on this topic.

As an aside to this thread, I'm about to pull the pin on an OPPO 105D and a NAS which will easily triple my overall LAN network traffic, as now video will be streaming from NAS to OPPO (both via Ethernet, through N66U, rather than the current setup of video from HTPC to TV via HDMI. I'm worried if the N66U is already struggling under the middling intense torrent load, will video streaming suffer? I haven't decided if I'll keep the HTPC and only torrent to it, or use the torrent client on the Synology NAS.

 

[jail4ceos2]

Sorry, maybe this got missed, so I'm going to bump it. I can't be the only one with these questions. If there is a thread already about this, it didn't come up in search.

 

[AndreaS]

Sorry, maybe this got missed, so I'm going to bump it. I can't be the only one with these questions. If there is a thread already about this, it didn't come up in search.

I'm not more using the N66 and, even at that time I used transmission/entware to torrent. Slowdowns and errors I experienced were all related to ram occupation. The larger the shared files were, the higher the possibility to run out of memory.

I'd suggest you to telnet/ssh to your router and try the 'free' command that shows you the memory usage and see if it is close to fill-up.

If that's the culprit, the solution is a swap partition or a swap file in an attached usb disk/pendrive or even in an internal sd/card. Pay just attention to the fact that continuous writes to flash devices wear them much faster than hard disks do.

 

[ryzhov_al]

Hi!

I think firmware is not so important there, they are all stable enough. This is firmware-independent kernel tweaks which I use:

# Bigger TCP receive buffer (default is 118784 bytes)
echo 524288   > /proc/sys/net/core/rmem_max
# Bigger TCP send buffer (default is 118784 bytes)
echo 524288   > /proc/sys/net/core/wmem_max
# Bigger RAM to keep free and untouchable (default is 2884KB)
echo 8192 > /proc/sys/vm/min_free_kbytes

"rmem_max" and "wmem_max" gets more throughput for torrent client. If you turn on Transmission debug messages, you may see following messages:

Nov 30 16:01:03 MBL3TB transmission-daemon[2888]: UDP Failed to set receive buffer: requested 4194304, got 65534 (tr-udp.c:75)
Nov 30 16:01:03 MBL3TB transmission-daemon[2888]: UDP Please add the line "net.core.rmem_max = 4194304" to /etc/sysctl.conf (tr-udp.c:80)
Nov 30 16:01:03 MBL3TB transmission-daemon[2888]: UDP Failed to set send buffer: requested 1048576, got 65534 (tr-udp.c:86)
Nov 30 16:01:03 MBL3TB transmission-daemon[2888]: UDP Please add the line "net.core.wmem_max = 1048576" to /etc/sysctl.conf (tr-udp.c:91)

"min_free_kbytes" forces LinuxVM to keep some RAM reserve. We can fall into following issue with default (too small!) value:

Oct  2 23:10:21 kernel: swapper: page allocation failure. order:3, mode:0x4020
Oct  2 23:10:21 kernel: Call Trace:
Oct  2 23:10:21 kernel: [<8000de20>] dump_stack+0x8/0x34
Oct  2 23:10:21 kernel: [<8005914c>] __alloc_pages+0x294/0x344
Oct  2 23:10:21 kernel: [<80077438>] __slab_alloc+0x1d8/0x4b8
Oct  2 23:10:21 kernel: [<80077a34>] __kmalloc_track_caller+0x144/0x184
Oct  2 23:10:21 kernel: [<801a8a30>] __alloc_skb+0x70/0x140
Oct  2 23:10:21 kernel: [<8020badc>] tcp_collapse+0x214/0x468
Oct  2 23:10:21 kernel: [<8020be80>] tcp_prune_queue+0x150/0x340
Oct  2 23:10:21 kernel: [<8020c778>] tcp_data_queue+0x708/0xe10
Oct  2 23:10:21 kernel: [<8020e594>] tcp_rcv_established+0x224/0xc00
Oct  2 23:10:21 kernel: [<802167dc>] tcp_v4_do_rcv+0xe8/0x450
Oct  2 23:10:21 kernel: [<80217598>] tcp_v4_rcv+0xa54/0xb38
...
Oct  2 23:10:21 kernel: Mem-info:
Oct  2 23:10:21 kernel: Normal per-cpu:
Oct  2 23:10:21 kernel: CPU    0: Hot: hi:   42, btch:   7 usd:   3   Cold: hi:   14, btch:   3 usd:   6
Oct  2 23:10:21 kernel: Active:14580 inactive:10447 dirty:1524 writeback:0 unstable:0
Oct  2 23:10:21 kernel:  free:2544 slab:1628 mapped:2695 pagetables:128 bounce:0
Oct  2 23:10:21 kernel: Normal free:10176kB min:1440kB low:1800kB high:2160kB active:58320kB inactive:41788kB present:130048kB pages_scanned:0 all_unreclaimable? no
Oct  2 23:10:21 kernel: lowmem_reserve[]: 0
Oct  2 23:10:21 kernel: Normal: 0*4kB 654*8kB 305*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 10176kB
Oct  2 23:10:21 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0

It happens when download speed growing too fast (for example, a local peer has been found) and gets higher then USB write speed. You can avoid it by increasing "min_free_kbytes" value, by giving LinuxVM more space to handle it.


The other torrent settings depends on your torrent client. IMHO, the common rules for weak systems is:

• Prefer unencrypted peers,
• Disable uTP protocol,
• Reduce per-torrent connections to ~10,
• Reduce overall connections to ~50,
• Reduce simultaneously downloading torrents to 1..3,
• Limit download/upload speed to 2..4MB/s.

Please keep in mind we have no EXT4 support (except wl500g project), so we can't minimize file fragmentation, which is important for embedded systems:
1. Highly fragmented file:

$ filefrag ./1080p_6000.mp4
./1080p_6000.mp4: [COLOR="Red"]2242 extents[/COLOR] found, perfection would be 8 extents
$ /opt/bin/dd if=./1080p_6000.mp4 of=/dev/null
1840984+1 records in
1840984+1 records out
942584097 bytes (943 MB) copied, 55,6186 s, [COLOR="Red"]16,9 MB/s[/COLOR]

2. The same defragmented file:

$ filefrag ./1080p_6000.mp4
./1080p_6000.mp4: [COLOR="Red"]12 extents[/COLOR] found, perfection would be 8 extents
$ /opt/bin/dd if=./1080p_6000.mp4 of=/dev/null
1840984+1 records in
1840984+1 records out
942584097 bytes (943 MB) copied, 43,4758 s, [COLOR="Red"]21,7 MB/s[/COLOR]

As you can see, read speed goes down 28% for fragmented file.

 

[Fab Five Freddy]

Hi!
Please keep in mind we have no EXT4 support (except wl500g project), so we can't minimize file fragmentation, which is important for embedded systems:

To minimize fragmentation, I've alway used

"preallocation": 2

According to the docs: https://trac.transmissionbt.com/wiki/EditConfigFiles#FilesandLocations
preallocation: Number (0 = Off, 1 = Fast, 2 = Full (slower but reduces disk fragmentation), default = 1)

It's slower to start up, but hey, I realise it's preallocating...

 

[jail4ceos2]

WOW, I'm going to have to try to use some of the low hanging fruit there, and then learn some of the more complicated stuff, but I do have one question. For some of those scenarios is the torrenting being done on the router itself, or on a device downstream on the network somewhere?

 

[ryzhov_al]

Router isn't a great seed box because of low I\O USB performance and very weak CPU (comparable to Pentium I). If you can chose, leave router for router's tasks.

 

[jail4ceos2]

So will the various solutions above be beneficial if I do torrenting on my HTPC? It happened again last night, the torrents slowed to 400kbs when they should have been maxing at 6Mbs from my private torrent site. When I rebooted just the N66U it jumped up to max. From what ryhzov says:

"The other torrent settings depends on your torrent client. IMHO, the common rules for weak systems is:

Prefer unencrypted peers,
Disable uTP protocol,
Reduce per-torrent connections to ~10,
Reduce overall connections to ~50,
Reduce simultaneously downloading torrents to 1..3,
Limit download/upload speed to 2..4MB/s."

I can only use about 1/3 of my total bandwidth. I have utorrent set to 999 total connections, 50 per torrent. And that isn't a solution.

Have I bitten off more than I can chew, as the code ryhzov showed I have no idea if I paste that in a single directory to modify the kernel or....??!