rtmp-services: Add encoder settings callback

Swaps out the old initialize callback code that applied encoder settings
to the new apply_encoder_settings callback