My frustrations as a network engineer

[Zargon]

I love 'the network must be broken, some of our PC's cant get on the network! ever since you rebooted the router!'



OH you have a switch downstream of my equipment?

you should check into that if machines from all 3 of your vlans can get to the router


OH "sorry we had SPAN configured wrong on one of our switches and it was blocking on the uplink"

 

[EricFiskCGD]

Not for nothing, I was on the front lines and it was horrible in ways I couldn’t imagine before I took this job.

The absolute worst was handling people with computer problems. I had to diagnose the problems over the phone and if I couldn’t fix it then I had to go there myself or dispatch someone else who was working the floors. Nine times out of ten someone either kicked out the network cable or installed something that they shouldn’t have.

One of the supervisors had a problem with his hard-drive, he couldn’t store any more information. I went and looked on his system and sure enough, his HD was almost full. I logged out of his account, logged back in using the Admin account and saw that his password protected folder had 8 gigs of the nastiest bondage and scat porn you could imagine.

The next day I was standing at his office door with his boss, The VP of his division, security and building maintenance to help him clean out his desk before lunch. I felt horrible for doing my job. It was my position to rat people out if they had porn, links to gambling sites, personal information… Some days I felt horrible because I helped someone lose their jobs.

Besides that, there were also horrible conflicts with-in the division; when people who know what they’re doing who have strong personalities clash with people who don’t know what the hell they’re doing but still have strong personalities, fights eventually break out. I’ve seen fights that could have gotten ugly if someone had a screw driver or box cutter in his or her hands!

 

[Gryz]

This thread is a good place to link The Profane Networker.

You might enjoy The Honest Networker as well.

 

[ibex333]

What I noticed even more that with these added complexity, the knowledge and networking troubleshooting skills on the customer side are pretty poor.

Heh.. You should be thanking your lucky stars their knowledge is so poor. It's because of such people that you still have a job.
Try finding a networking job in New York City. ; )

 

[alkemyst]

I can't print!

Not a network problem.

It's never a network problem. Ever.

My application isn't working, must be a network issue!

Funny, it's only your app that is having trouble. Not a network problem.

A blown JetDirect card is indirectly a network problem.

 

[Zargon]

A blown JetDirect card is indirectly a network problem.

end user hardware failure

a printer is not a piece of networking hardware

its access layer gear

nice try though

 

[Zargon]

this is why we cant have nice things people


project I completed last week, refreshed all the equipment, and rewired the whole thing.

added a 10gb uplink to our backbone

 

[alkemyst]

end user hardware failure

a printer is not a piece of networking hardware

its access layer gear

nice try though

With Ethernet makes it part of a network. Depends on shop who solves the issue.

Nice try telling that to a C type.

 

[Zargon]

With Ethernet makes it part of a network. Depends on shop who solves the issue.

Nice try telling that to a C type.

just because it has ethernet doesnt make it part of the network

user devices that hang off edge ports are NOT part of the network

type C? pretty slim on details there for part C if your rationale is that it runs a networking protocol means its part of 'the network'. nice 10000 ft view. personalilty type T for troll maybe?

who solves the problem really doesnt tell you jack either, what creates the problem doesnt necessarily tell you that either

security finds computers having problems that are spewing malware, end user machines, that security has no control over, is that now a security problem?

its spamming the network with malware, is it now networkings fault if we notice a link saturated and find a user PC is spamming malware?

no. because that doesnt make sense.


I did a building upgrade a while ago, and a user PC FREAKED OUT and was spamming broadcast traffic out its gig interface and it was a multibuilding subnet, so it saturated the uplink. the departments DNS and DHCP servers were in the building, but no one could talk to them due to the broadcast storm. some piece of software on it flipped because it couldnt find its gateway and decided to really really try to find it.

not a 'network problem'. user created issue.(the department is the user in this case.)

its impact was amplified due to poor design. (we have tried to get them to move to one user subnet per building, and move their DNS/DHCP to one of our colocation centers where we have far more redundancy, UPS's, generators, etc, but they wont do it)

 

[imagoon]

Jetdirects for us are mostly the printer guys. Only time the network team is involved is to plug the MAC address in to DHCP or if something on the network is actually causing the issue (really rare except when end users add local switches.)

 

[alkemyst]

just because it has ethernet doesnt make it part of the network

user devices that hang off edge ports are NOT part of the network

type C? pretty slim on details there for part C if your rationale is that it runs a networking protocol means its part of 'the network'. nice 10000 ft view. personalilty type T for troll maybe?

who solves the problem really doesnt tell you jack either, what creates the problem doesnt necessarily tell you that either

security finds computers having problems that are spewing malware, end user machines, that security has no control over, is that now a security problem?

its spamming the network with malware, is it now networkings fault if we notice a link saturated and find a user PC is spamming malware?

no. because that doesnt make sense.


I did a building upgrade a while ago, and a user PC FREAKED OUT and was spamming broadcast traffic out its gig interface and it was a multibuilding subnet, so it saturated the uplink. the departments DNS and DHCP servers were in the building, but no one could talk to them due to the broadcast storm. some piece of software on it flipped because it couldnt find its gateway and decided to really really try to find it.

not a 'network problem'. user created issue.(the department is the user in this case.)

its impact was amplified due to poor design. (we have tried to get them to move to one user subnet per building, and move their DNS/DHCP to one of our colocation centers where we have far more redundancy, UPS's, generators, etc, but they wont do it)

I guess you aren't working to high up the chain or in big business...C types are your CEO, CCO, CTO, CFO, etc.

When the shared printer stops working and they call you to fix it, unless you absolutely have another crisis going on you don't go giddity giddity giddity and tell them that's an end user device, click.

 

[Mushkins]

I guess you aren't working to high up the chain or in big business...C types are your CEO, CCO, CTO, CFO, etc.

When the shared printer stops working and they call you to fix it, unless you absolutely have another crisis going on you don't go giddity giddity giddity and tell them that's an end user device, click.

No, you confirm that it's a dodgy JetDirect card causing a broadcast storm and forward the case over to desktop support because replacing printer hardware and configuring the printer beyond assigning it a static IP address is not the job of a network engineer. It's desktop supports responsibility to work with the printer vendor to get it fixed (or fix it themselves).

 

[imagoon]

I guess you aren't working to high up the chain or in big business...C types are your CEO, CCO, CTO, CFO, etc.

When the shared printer stops working and they call you to fix it, unless you absolutely have another crisis going on you don't go giddity giddity giddity and tell them that's an end user device, click.

No, we call the printer support guys and let them deal with it.

 

[alkemyst]

No, you confirm that it's a dodgy JetDirect card causing a broadcast storm and forward the case over to desktop support because replacing printer hardware and configuring the printer beyond assigning it a static IP address is not the job of a network engineer. It's desktop supports responsibility to work with the printer vendor to get it fixed (or fix it themselves).

It's a service outage and none of those C types can print on Monday morning.


I guess I work higher up the chain or maybe I just solve the issue faster than the help desk.

I get were you worker bees are going with this though, but good luck telling the guy that signs your paychecks to go pound sand when you are just sitting playing Minecraft.

 

[Zargon]

I guess you aren't working to high up the chain or in big business...C types are your CEO, CCO, CTO, CFO, etc.

you think pretty highly of yourself there don't ya? I pointed out how much you lacked attention to detail in your diagnosis to call yourself type C, but I guess that's your type C personality coming out again

When the shared printer stops working and they call you to fix it, unless you absolutely have another crisis going on you don't go giddity giddity giddity and tell them that's an end user device, click.

I LOL that the helpdesk put the case through to networking and have it reassigned to the correct department so the right person fixes it.

if its causing network issues because it freaked out, I will disable the port, and note on the ticket to contact us when their hardware issue is resolved and we will turn the port back on.

If the CIO walks down the hall from his office to mine, sure, I'll help him out, hes the freakin CIO afterall , but he's not an idiot, and he probably came down to ask me because he cant get ahold of the desktop/printer guy and needs something right now. so your situation is still pretty lacking, might as well add in a sharknado to the mix to make it slightly MORE probable

we also don't put our Cx0's on big shared printer pools generally, and I imagine the CIOs office admin assistant would be taking care of contacting support anyways



bottom line, you are reaching and it doesn't take a type C personality to figure it out

 

[imagoon]

It's a service outage and none of those C types can print on Monday morning.


I guess I work higher up the chain or maybe I just solve the issue faster than the help desk.

I get were you worker bees are going with this though, but good luck telling the guy that signs your paychecks to go pound sand when you are just sitting playing Minecraft.

We still would turn it over to the printer guys. Hell after asking the question, my CIO would be like "ok" and use another printer. It isn't like there is only one on the floor. Then again the C types I work with actually call the helpdesk and let the helpdesk install another printer for them or actually read the document that is on the desktop in the image that is named "how to add printers" and do it themselves.

We are actually going through this right now. We a common copier / printer that is roached that is over in the "C wing". Other then a question from the team for me to "please check the print queue / ping it" the printer guys and helpdesk have taken it and other than the stream of tickets in the ticketing queue, I have nothing else to do with it.

 

[Zargon]

It's a service outage and none of those C types can print on Monday morning.


I guess I work higher up the chain or maybe I just solve the issue faster than the help desk.

I get were you worker bees are going with this though, but good luck telling the guy that signs your paychecks to go pound sand when you are just sitting playing Minecraft.

you are such a troll dude


backpeddaling all the way

you 'work' higher up the chain?

first it was a dead jet direct card

now its a full on service outage




so is it common for your c types to have no idea how to get a problem fixed other than grab the first guy they find? sounds like an awesome place to work

 

[Zargon]

We still would turn it over to the printer guys. Hell after asking the question, my CIO would be like "ok" and use another printer. It isn't like there is only one on the floor. Then again the C types I work with actually call the helpdesk and let the helpdesk install another printer for them or actually read the document that is on the desktop in the image that is named "how to add printers" and do it themselves.

We are actually going through this right now. We a common copier / printer that is roached that is over in the "C wing". Other then a question from the team for me to "please check the print queue / ping it" the printer guys and helpdesk have taken it and other than the stream of tickets in the ticketing queue, I have nothing else to do with it.

word.

someone would try and ping it, and track down the jack to a switch port and make sure its not a network issue, when there is no link and its on the correct vlan, someone is calling the system/printer guy to expedite it. if its slow we might even look at the actual printer for a second, or if ya know, its some big thing that's off to the board of trustees, but that's not what this started as, he keeps trumping up the problem trying to be right

 

[alkemyst]

I get it guys...you are freaking nuts.

Zargon, I never called myself a C type. No C types just don't grab the first guy they find.

You are spinning some kind of story I never told.

Seek medication.

 

[alkemyst]

you think pretty highly of yourself there don't ya? I pointed out how much you lacked attention to detail in your diagnosis to call yourself type C, but I guess that's your type C personality coming out again



I LOL that the helpdesk put the case through to networking and have it reassigned to the correct department so the right person fixes it.

if its causing network issues because it freaked out, I will disable the port, and note on the ticket to contact us when their hardware issue is resolved and we will turn the port back on.

If the CIO walks down the hall from his office to mine, sure, I'll help him out, hes the freakin CIO afterall , but he's not an idiot, and he probably came down to ask me because he cant get ahold of the desktop/printer guy and needs something right now. so your situation is still pretty lacking, might as well add in a sharknado to the mix to make it slightly MORE probable

we also don't put our Cx0's on big shared printer pools generally, and I imagine the CIOs office admin assistant would be taking care of contacting support anyways



bottom line, you are reaching and it doesn't take a type C personality to figure it out

Cxill bro. You are going above and beyond in your lack of knowledge of real life. I am guessing you are in your 20's. If not that's sad.

 

[Zargon]

With Ethernet makes it part of a network. Depends on shop who solves the issue.

Nice try telling that to a C type.

Zargon, I never called myself a C type. No C types just don't grab the first guy they find.

You are spinning some kind of story I never told.

Seek medication.

You were either calling yourself one, or changing the situation to make yourself right.

at first I thought you were implying you were a c type personality, ie detail oriented




all this other BS aside, a blown jet direct card is NOT a network problem. period. ever. its failed user/access level equipment. whether that falls to the 'networking group' or network guy, is up the org.

my last job, I was one of two IT people, so it would have been my job, as was everything, now its not. because I am in the networking group. we have another group for that, and I would even be able to get to whereever they might be storing spare jet direct cards. because we dont share storage space.

Cxill bro. You are going above and beyond in your lack of knowledge of real life. I am guessing you are in your 20's. If not that's sad.

 

[Mushkins]

It's a service outage and none of those C types can print on Monday morning.


I guess I work higher up the chain or maybe I just solve the issue faster than the help desk.

I get were you worker bees are going with this though, but good luck telling the guy that signs your paychecks to go pound sand when you are just sitting playing Minecraft.

Separation of duties is a pretty common business structure concept... If you're in a small org you might be the network guy *and* the printer guy, but then it actually would be your job to fix it.

I don't know where you think anyones saying to tell the executives to "go pound sand" while we sit on our thumbs and play minecraft. Anything but, we do our jobs. We go above and beyond our role, tell the CEO we'll see whats going on and make sure it gets to the right people to take care of it, make sure the ticket gets generated if they didnt call it in and routed to the *right* group, and maybe call a desktop guy directly and say "hey, the CEO just stopped by to bitch about this printer being down, you guys might want to expedite this one before the shit rolls downhill onto you, here's the ticket #." Then we go back to doing actual network engineer work like configuring switches and troubleshooting whatever is wrong with Vlan6 not properly QoSing VoIP traffic in XYZ office making everyones phone calls sound like a 90's flip phone.

 

[wizdum]

This is a great thread I'm loving the stories. One thing I hate as a network engineer, is that you're supposed to know everything! I'm just the type of person that if there is someone more knowledgeable around, I will ask for help to save time or to solve a problem I'm stumped on. A lot of times, corporate higher-ups will look at this as you don't know what you're doing.. but if I've learned anything in this industry, it's that no one knows everything. Everyone googles.. and someone who is in the office next to you with 20 years experience over you, is probably worth a shot to stay efficient on time.

 

[imagoon]

This is a great thread I'm loving the stories. One thing I hate as a network engineer, is that you're supposed to know everything! I'm just the type of person that if there is someone more knowledgeable around, I will ask for help to save time or to solve a problem I'm stumped on. A lot of times, corporate higher-ups will look at this as you don't know what you're doing.. but if I've learned anything in this industry, it's that no one knows everything. Everyone googles.. and someone who is in the office next to you with 20 years experience over you, is probably worth a shot to stay efficient on time.

We were just discussing this a few weeks ago. Specifically Sharepoint. Sharepoint Admin may not be a Sharepoint Developer or vis versa. We have a Sharepoint admin here that is "sharepoint admin" in that he manages SQL / some of the apps. He never claims to be a 'pro admin' or a developer but people are always asking for development / webpage design. When he says, "well we would need to get a quote" people look at him like he has no clue and how is he "managing" this system. Yet another one of those "IS vs IT" issues. Dev work = IS while making the platform work tends to be more IT. Some people really don't grasp the difference.

 

[RFC Rudel]

I am a mcp/mcsa/mcse/mcse security since 1999

Mcse on nt4/2000/2003/security/


in 15 years working in a consulting firm I migrate from banks to small business, taking roles from design to deploy and troubleshoot.(my specialty is troubleshoot)

My experience allow me to manage several technologies, desktop deploy/AD/Networking/Security/Exchange/clusters.

many MCSE lack the hardware or experience to troubleshoot, and the certifications are so so, because a product like SharePoint use a lot MS technologies that you need to master, or all of the problems of MS in the same product.......( I was in charge of a project using the first SharePoint back in the days......it was terrible)

Troubleshoot require that you must now many layers, from hardware/networking/security etc.

I have seen many mistakes and recommendation from MS consulting or others that was wrong because they don't think of entire picture. Hardware is the primary hole you find today in consulting services.

yesterday I was explaining to a coworker that ask why X server have X type of storage etc. and I say that you must have in mind the storage needs down to how the raid card was connected to the mother, (raisers, chipsets, internal latency in the buses etc.)

Some companies representatives try to sell any type of hardware.....or the support that BIG server makers have here in Argentina are a total joke ( 2 INCIDENTS with servers on 2 sites, same servers, same problem (bios issue whit raid controller), on 1 site they blame the electrical installation, the other recommend to install enterprise version of windows server....)

Another problem that the creator of this tread did not have to fight is developers....

the best developers I find use to work in complex network/software support jobs, to find one that take security/performance and how will impact to core services or ask about it is hard to find.

in a network that I manage I have constant problems with the sql developer (is my friend) he do wonders in sql but he don't understand crap about the layers he jump to get his work done, he is very smart EJ: to integrate SQL and Exchange to do data mining on Exchange databases his developers skills hit a wall (I am the wall) if he hit the mail servers they will hit the AD controllers etc., and is an ongoing fight.
Same as security and usability, more security lees usability (and works the other way around)


Today the cloud, the cloud, the cloud, yes send you data to the cloud (cloud because you don't have a clue of where it is and who has access)

When you share your data whit external services you share that company risks (angry personal, poor quality personal etc.)


Not to mention that Microsoft become the competition, many Exchange admins are not happy. I understand the cloud to small business, and I have no problem if one client migrate, but if you used the Microsoft recommendation to many clients to use exchange......and now.....Microsoft say......(I want my certification $ back )


IT Admins, the better you do your job the less they think they need you....



ALL
PEOLE
SEMS
TO
NEED
DATA
PROTECTION