Odd.
First, I wouldn’t turn off the WiFi at all, on either device, the system will normally choose its own ‘best path’ for the data.
Second, is there an issue with the cable you’re using for the line in? I have (most unfortunately) a few mono 3.5mm cables, and the visual difference is slight at best. I’m trying to rid myself of them as I find them, but this is a common issue for me. I’d also want to be sure that the cable is inserted properly into the back of the Sonos Five, sometimes it seems hard to seat properly.
Finally, have you submitted a system diagnostic within 10 minutes of experiencing this problem, and called Sonos Support to discuss it? If there is a hardware issue, I’d think it would show up in a diagnostic, but I’m not 100% sure.
There may be information included in the diagnostic that will help Sonos pinpoint the issue and help you find a solution.
When you speak directly to the Support staff, they have tools at their disposal that will allow them to give you advice specific to your network and Sonos system.