Which part is your problem, serving the media from disk, or transcoding and serving that stream?
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
I've updated the OP to answer this. I think serving the media from the spinning disk is the heart of the problem.
Prove it with data, else you'll just be blindly throwing money down the drain.
The network testing I’ve done (iperf and file transfers) hasn’t revealed any issues. I’m seeing consistent 1Gb speeds. I could try some wireshark monitoring, I guess.
I believe I have a very similar issue as you. It only occurs on remux/high bitrate 4k videos. Happens no matter the player etc. I currently fix this issue by putting on that limit qbit upload speed and everything plays fine then.
I am interested in what your solution is.
limit qbit upload speed
Not sure what you’re referring to here.
Its a qbittorrent setting. you can set upload and download limits to turn on at specific times or when you want to by clicking the little speed dial.
So if you are not utilizing those harddrives for other uses such as torrents thn disregard.
Got it: qbit == qbittorrent. I've thought about getting the remaining docker containers off of the Synology. I may look into that this weekend.
Yea I would test a large file and as soon as it buffers put a limit on quittorrent or something that could be reading and writing lots of data from those drives and see if that fixes the issue. I wish there was a better way.
I just did something sort of like what you are doing and after a few hiccups, it's working great. My Synology just couldn't handle transcoding with docker containers running in the background.
Couple differences from your plan: I chose a N100 over the N150 because it used less power and I wasn't loading up CPU dependent tasks on the thing. The N150 is about 30% faster if memory serves, but draws more power. Second, do you really need a second m.2 SSD BTRFS volume? Your Synology is perfectly capable of being the file storage. I'd personally spend the money you'd save buying a smaller N150 device on a tasty drive to expand the existing capacity then start a second pool from scratch.
Finally, I wouldn't worry about converting media unless you are seriously pinched for space. Every time you do, you lose quality.
just get a cheap n100 box, don't overspend
I had issues streaming directly from one device to the other without transcoding on WiFi. (I know you're wired! Heard me out.)
I found that, although it didn't fix the issue, it did help to switch from using SMB to NFS. Something about the way the protocol works meant that SMB had enough of an overhead that it worsened my stuttering issues outside of the spotty WiFi connection. For sure it significantly sped up scrubbing access times as well.
It may not be the issue, but it may be a step worth checking just to see if it is a part of the issue.
For what it's worth, 4k remuxes can have bitrate spikes well exceeding the limits of a single gbps wire. If you have a player with limited memory, or just limited cache settings, this may also be a part of the problem.
I’ve looked into NFS multiple times. I work in HPC implementation and believe me I know about SMB/CIFS performance (or lack thereof!)! I just haven’t had the time to figure out ID mapping. What NFS version do you use, and how do you handle file ownership on the shares? I suppose it’s all read-only, so that would make it a bit easier?
It's all read only, yes, but I just use a group specifically for NAS access and put users that need it in there.
I use the NFS version from the debian repository; not actually sure which one, and didn't even know that it mattered.
Yeah, nfs v2 3 or 4 can make a difference. I don’t know that many use v2 anymore. If you’re using the current release in your distribution and didn’t specify a specific version, I would guess you’re using v4.
I would ideally like to convert the library to h.265 or even AV1 if I can make it work.
Unless you've downloaded remuxes (which I doubt), I'd seriously recommend redownloading instead of converting your existing files.
h.265 and especially AV1 take a long time to encode by CPU, and hardware encoding won't give you any space savings, unless you're okay with losing much details.
Redownloading is most definitely faster, will result in more space savings for the quality you'll get. PS: Unless you've got data volume limits, but even then I'd recommend slowly upgrading over time. It's quite simple with TRaSH guides and giving h.265 a higher score.
Agreed! The library is populated by my parents and me. I’d have to look again, but i think it’s around 12TB.
TBH I would consider an M4 Mac Mini and either a NAS or DAS to go along with it. The power, efficiency, and price make it a really compelling choice.
I did this. It is a mixed bag.
I have thought of that... Most of the daily driver systems in the house are Macs, so it would probably work out pretty well.
If you went from "everything works", to "now it stutters", then you either have a networking issue, or a resource issue on the source.
Did you update something recently?
Do you have network stats from your router?
Do some devices work fine, and others don't?
I see no other issues from the network. The thing that “changed” is me trying to watch 4k stuff on my plex server. Up until recently, I didn’t bother with 4k. No real reason for trying it now, TBH. I’ve never felt that 4k was necessary for home viewing on anything smaller than a 100” screen (my largest if 75”).
Okay, so what's your network look like?
Specs on your router, is this wired or wireless to the Shield, how much other traffic are the other network clients pulling, and is this a constant, or just happens intermittently?
Everything's wired. Router is a TP-Link BE63, 2 APs w/ wired backhaul. Shield is on the same switch as the synology. STBs are throughout the house, but generally max 3-hops to the Shield/Synology. All Netgear bluebox 1Gb dumbswitches. At some point in the near future, I plan on getting this stuff to a central switch, so everything is a leaf switch away from it.
ETA: if I’m watching something, the network is generally pretty quiet. I have most data-intensive things (downloads, backups, off-site replications) set to happen in the wee-hours.
Netgear is...kinda shit. Is that 1Gb for the whole switch, or per-port? Any traffic steering on the network?
They’re all just dumb consumer switches. Nothing managed… yet.
I would double and triple check your not transcoding. Even if you're watching on the client at whatever the files native resolution is depending on the codec of the file you might still be transcoding. For instance with 1080p anything h265 or AV1 is transcoded into h264 by the server. There's also a few other situations where Plex with force transcoding or down convert the video whether you want it to or not.
Your NAS shouldn't be having trouble serving the file to Plex I'd bet it's transcoding in the background and you just don't realize it.
Aside from looking at the current activity on the server web page, where might I look to see if this is true?
Every Plex client is a little different, but there is usually a video details or "playback info" button that will give you stream info such as direct play, transcode, or transcode (HW) for hardware support.