cross-posted from: https://lemmy.cloudhub.social/post/347779

I am running a Kubernetes cluster for this domain, and I’m looking at more services to run (right now I have Mastodon and Lemmy).

I was considering WriteFreely and PixelFed, but they don’t seem to have an easy solution for running on Kubernetes (WriteFreely doesn’t even have a production-ready docker image).

Is anyone else running federated services in their lab? Do you run any of them on Kubernetes?

  • seang96AEnglish
    2·
    2 months ago

    Using different federation protocol, but matrix wservers ould be the other big one.

    Edit you also mentioned trouble creating them. I suggest looking into operator hub and using operators for postgres and redis and auth (keycloak?). This can get you down in the rabbit hole for making everything highly available too.

    • jax@lemmy.cloudhub.socialOPEnglish
      2·
      2 months ago

      Yeah, I used to host a Matrix instance - could do that for this one too.

      The issue is more about setting up the Kubernetes manifests and templating them. I usually use the chart’s built-in postgres and redis config, though using an operator would make it more scalable for sure.

      I’m using Authentik for auth, but I do also like Keycloak.

      • seang96AEnglish
        2·
        2 months ago

        Yeah it’s a bit of work sometimes. Synapse matrix kinda sucks too their philosophy of no environment variables for secrets. I ended up making an init container that hijacks my config map and I jet’s the environment variables into the config

        • jax@lemmy.cloudhub.socialOPEnglish
          1·
          2 months ago

          They store the secrets in a file? Gross. What a poor way of handling that. Pretty sure environment variables would be more secure. Especially in Kubernetes.

          • seang96AEnglish
            1·
            2 months ago

            Yeah I want to switch when other implementations catch up. Unfortunately I think that will be some more time especially since you can’t migrate from synapse and have to start from fresh. One day though!

            I did the same for Lemmy at one point then found out all the configs are mapped to environment variables my convention. My Lemmy setup is the most advanced, but it has HA postgres, and all of its modules separated and HA. The proxy setup for it in k8s was rough but I eventually got it working directly on ingress-nginx too.

            • jax@lemmy.cloudhub.socialOPEnglish
              1·
              2 months ago

              Huh, do you have your lemmy config documented somewhere? I keep running into issues with it and I’m not sure which component exactly is failing, but it’s annoying. I’m using this helm chart currently: ananace/lemmy It works, but I don’t have pict-rs setup in HA either.

              • seang96AEnglish
                1·
                2 months ago

                I got all my yaml files source controlled privately right now but I can share if you want them. I disabled Pictrs around the time of CSAM attacks and have yet to bother enabling it again haha

                • jax@lemmy.cloudhub.socialOPEnglish
                  1·
                  2 months ago

                  I disabled Pictrs around the time of CSAM attacks and have yet to bother enabling it again

                  Uhh… what?? When did that happen? I thought pictrs was a requirement also…

                  • seang96AEnglish
                    1·
                    2 months ago

                    Nah not a requirement. I think like 3 months or so after the reddit API shutdown. Big instances got local AI models to detect it and Lemmy server now supports disabling caching other instances so I’d probably disable that if I ever enable it again haha