Beyond Grids: Scaling up Continuous Kernels via Adaptive Point Representations for Sensor-Based Human Activity Recognition