tests: Move chalice under toxgen #3360

Job	Run time
Flags (pinned) (3.9, ubuntu-22.04)	1m 44s
Flags (pinned) (3.7, ubuntu-22.04)	1m 35s
Flags (pinned) (3.8, ubuntu-22.04)	2m 11s
Flags (pinned) (3.12, ubuntu-22.04)	3m 32s
Flags (pinned) (3.13, ubuntu-22.04)	4m 7s
All pinned Flags tests passed	2s
	13m 11s

Provide feedback