Ask Lemmy
A Fediverse community for open-ended, thought provoking questions
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, and toxicity are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com.
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.
6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !askusa@discuss.online
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
view the rest of the comments
This was recent, and it's still pretty sore for me. I doubt anyone will be able to pinpoint who I am, but if you for some reason are in this forum and recognize me please DM me. Try to count all of the red flags.
I was hired as a software engineer and was immediately thrown onto a "high-visibility project". My service was the middle man between two other mission critical services. Essentially downstream provided metrics and needed to get to upstream.
I laid out several different architectures that I recommended. First was prometheus. It's literally designed to do this, downstream is spread across many servers, prometheus is literally built to do this. Upstream then can scrape prometheus, any other future dependents can also scrape. This was rejected. "We did prometheus once, it didn't work." I check, it's a single tenant instance of Prometheus running on one 24XXL AWS VM. So, they didn't know how to properly configure prometheus. I tell them I can kill 2 birds with one stone. No, prometheus bad. Rejected.
Second, we use a highly reliable queue setup. Downstream publishes to queue, Upstream reads from queue. Seems simple enough, can have many producers and many subscribers, and we already have a kafka service. Rejected. For why, I ask. Literally "Upstream doesn't know how to work with queues". Literally got that as an answer. Read that as "We need to choose a subpar architecture because we openly admit our engineers don't have the necessary skills". I even offered to help them, to write that part of the code. Rejected.
Third option came straight from the CTO. We love datadog here. Everyone does. Datadog. Oh you feel that pit in your stomach don't you. The mandate came down from on high that Downstream would push metrics to Datadog. I then would need to periodically scrape Datadog, and then have an API that the upstream could then periodically scrape me. I looked into Datadog's API. They don't really support this. I reach out to Datadog, talk to their engineers, and they confirm this is a horrible pattern. I bring this up, say it's just not a good decision, there are better ways. Literally rejected by the CTO himself.
So, I build this rickity ass service, brand new built with thumbtacks and glue. Along the way more is mandated to me. We'll have literally 8x the number of metrics we originally planned for. We're well over Datadog's API limitations. I am mandated to put it into a Postgres instance. Every decision I am overridden.
On top of this, Downstream is completely overworked and doesn't have time to answer questions about specific metrics. Upstream then asks me, who has been there now for a grand total of 4 months, and I don't know the specific questions. I refer them to Downstream for helping describe what specific metrics are and do. They report to my superiors that I am not being a team player for this. They also don't know how to use my API, I have to explain concepts like GET and POST to them, how to serialize datetimes. I end up writing some of their code for them just to make it work.
In the end, we shipped late. There was an arbitrary deadline set by the CTO that we missed - we were not consulted on this deadline, there was no reason for the deadline beyond "We should be live on this date". We missed it by 5 days. During those 5 days I am online every waking moment, sleeping an average of 4-5 hours per night. I'm a walking zombie trying to patch this thing.
A week after release I'm called onto a meeting with my direct boss, who reports to the CTO. He tells me that due to my "Lack of Ownership" and "Lacking team spirit" they are letting me go. I'm stunned. This entire time literally any decision I tried to make was overridden. They chose the worst possible architecture, forced me to implement it, forced me to talk to third parties about designing this anti-pattern. I had 2 other teams actively work against me, and on top of that I had no support from anyone. I was alone, and isolated. I got off that call, and I just cried. I felt like such a failure.
I'm at a new job now, and I've realized what a toxic environment that was. Horrible engineering practices, way too much pressure on me alone. I had developed health issues that I wasn't even aware of that now have subsided. I literally tried my best, and they just let me go. I found out later that my boss who fired me was being chewed out over the horrid project, and he put 100% of the blame on me to save his own ass.
Thanks for listening
A nightmare. I've never had anything quite that bad, but I've had plenty of work experience where management is making bad decisions and has no accountability.
My current role is hourly, so I'm happy to shut the laptop exactly 8 hours into the day. Pays a lot less, sadly.
That sucks. It sounds like a dodged bullet and I wouldn't take it personally.
I've seen this happen from different angles where a manager or c-suite has them hanging people out to dry so they can protect their own ego. Even if you managed to get through that situation, you'll feel like you're walking on glass for every project and that's just not a way you want to live.
When this job is frustrating, it can be the most frustrating of any type of job.
Sorry and thank god you got out.
Jesus fucking christ, that's incredible. I don't know what any of that software engineering stuff is, but based on your description of events, it just sounds like a waking nightmare.
If it makes you feel better, companies like that where upper management are quick to throw people under the bus for their own fuck ups are usually not long for this world. They'll continue to make bad decisions and the effects will be felt both downstream with low employee morale and retention rates, as well as commercial loss for the company. You are much better off being valued for your work elsewhere.
I did a lot of consulting. Companies like that include well-respected, well-known firms. They are so profitable that they can afford to have utter shit management in non-core functions.
Oh man, i felt that one. Thats full on red flags and if i were in your shoes, and being fired didnt stun me, id have sent a mail to all teams and managers involved (including ceo and cto) and shown the emails and reports showing the toxicity. Wouldnt have gone down without a final fuck you haha.
That said, sometimes its best to let it go and start something new somewhere else, like you did.
Im also in a semi similar scenario at the moment. They want me, and 2 others, to make a new version of their program in .net. Thats all fine and dandy. We get functional input when asked and all that.
We chose to create .net code/windows that are executed/called from inside the old client. However, when push comes to shove there is no support whatsoever. The other client team doesnt want to implement our stuff and delays it as much as they can. The server /api team does whatever the fuck they want, constantly breaking everything and choosing anti-patterns on their end. Manager will (and suspect he has already) thrown me and the architect under the bus because we have exposed the bullshit of said server team.
Besides functional input we are completely on our own with no support, on the contrary. We have to fight for every choice or design. And god knows when our code will get released... Its been 3 fucking years and our code has been done for a very long time.
Hell, even release was hard. "Can i push release from devops pipelines?" "No."
"Can i execute the needed script automatically from devops?" "No"
"You want me to do release manually?" "Yes". " not going to happen mate".
I also made several packages and helpful tools to make communication with the system easier, but dont you dare think anyone has promoted that internally to other teams. Hell, when i do get feedback and report that, nothing gets done with it. No ask or preasure to server team to implement the api calls.
So many issues they could solve if they just fucking listen
Id send the mail i was talking about here too when i get out, but im a consultant. I should not throw my company under the bus for this when we find a new project for me to do but ive been really badly wanted to do it and have the ceo in cc because fuck that
People like you and i should not doubt ourselves because of shit like this. From your story you know your stuff and listen to what the team has to say, you consider their experiences and what they are prepared to do. Thats good skills for an architect!
Know that I completely empathize with everything here, even the consultancy bits. It kills me when companies won't do basic things like CI/CD. In fact, that's one of the major red flags. If a company isn't minimum doing some sort of pipeline to go to prod it's pretty much a no from me at this point. It's such an important critical security step that if you didn't do that, you're not doing a lot else either.
Sounds like you're where I was, but you've had a longer timeline luckily. Don't let them blame you, don't let them put it on you. Sounds like you're walking that political tightrope pretty well.
Thank you for the kind words, I've been feeling pretty low since then and your last few words made me smile :)
This is a bigger and bigger red flag as I gain experience.
That, and neglecting the absolutely most important part of the app - data structures and their relationships. Including a deliberate level of normalisation.
Just shoving JSON into Mongo/DynamoDB means that you haven’t thought things through enough.
There are times when stuffing json into Mongo is really all you need. I remember them well because of their rarity.
True and well articulated.