Yeah... but Apple Silicon requires firmware for a lot of the peripherals. The upstream Linux kernel will now happily be able to load said firmware into the right place... but it doesn't come with it, so don't expect it to be as easy as rebuilding the kernel and dropping into a generic ARM image.
That's why Asahi Linux has a custom installer. Amongst other tasks, it installs a stub macOS partition that contains known supported Apple firmware versions. See "Whats with all the disk space?" under the FAQ at https://asahilinux.org/2022/03/asahi-linux-alpha-release/