Services Impacted:
- Switch (All) Preparation Queue
- Switch (All) Trade Queue
- 3DS (All) Trade Queue
On December 11, the team discovered an issue causing abnormally high disconnect rates with our customers. There are two network infrastructures powering Rawkhet Pokemon – an isolated local WiFi network which is used in HQ and a Personal Area Network (aka mobile hotspot) used on-the-go.
For context, preparation is only done in HQ and trades done on-the-go. This worked fine during the 3DS era. For the past 3 weeks, most trades have been done in HQ as well. However, we had to conduct most of our trades on-the-go today.
Pokemon Sword and Shield require far more stringent NAT (Network Address Translation) compatibility compared to previous games. Incompatible NAT Types such as B-D, C-D, or F would result in trades failing to connect. We have discovered that the PAN setup we were using has a undesirable NAT Type of D (HQ uses B). This was likely caused by Carrier-grade NAT (CGN).
To prevent additional frustration and errors we have taken services offline for the night and will resume in the morning. We have also just invested and implemented a new dedicated PAN solution overnight that resolves the NAT issue, with a dedicated trunk for Rawkhet services.
Apologies to all customers who were affected by the delays today!
