Okay web server/network/Linux gurus, you up for a challenge?

[Russ]

I've been beating on my server setup for a few weeks and trying to get my block of IP addresses to work. I've now determined WHY they don't work, but have so far been unable to figure out how to solve the problem.

Here's the setup:

The net pipe is a Cisco 675 DSL modem/router. This plugs in to a Linksys 5 port switch. From this switch, one patch goes directly to Eth0 on the server, and a second goes to a Linksys DSL Router. All other systems on the network are using the Linksys DSL router for web access and DHCP.

This setup is supposed to bypass the Linksys DSL router for any requests to the IP addresses assigned to Eth0 in the server.

Because a picture is worth a thousand words, and I realize I may not be adequately explaining this, I diagrammed my network. Diagram is HERE.

Here's the problem. If you go to http://207.108.218.137 the server loads up lickity split. This IP address is assigned to the Linksys DSL router and is port-forwarded to the second NIC on the server (Eth1). With me so far?

The other IP addresses are 138, 139, 140, and 141. These are all assigned as virtual IP's to Eth0 on the server. BUT, what is happening is that if you ping or traceroute to any of them, they all respond with 137, the address that is assigned to the router and port forwarded.

So, the requests are NOT bypassing the Linksys DSL router as they should. The question is how do I solve this problem?

All hardware verifies good. Everything on the network communicates with everything else and the web no problem. Apache is working just fine on all addresses as internally ALL of them load up fine. But, the only one that works externally is 137.

Ideas? Thanks.

Russ, NCNE

PS: This is the same post that's in the network forum, just wanted to make sure I didn't miss the brainpower that hangs in here.

 

[Rendus]

On the Linksys, do you have ICMP forwarding enabled? It's probably taking over the ICMP duties as most of the time, NAT breaks ICMP, but I THINK (not sure) there's an option for it somewhere.

-Edit- Er. Thought it didn't apply, then thought it was the Cisco doing the NAT, then kicked myself and realized I was looking at it the right way the first time

Strange network config, though

 

[Russ]

<< Strange network config, though >>



Hehe. Yeah, I'm surprised I got it all working. I'm not exactly an expert at networking.

The Linksys DSL router doesn't do ICMP forwarding as near as I can tell; nothing in the setup interface, anyway. The only mention of it in the docs for the Cisco 675 is to outline what the acronym stands for, and to say that it's actually part of the IP protocol.

It's Voyager night, so I'm headed home, but I'll check back tomorrow. Muchas gracias.

Russ, NCNE

 

[divide by zero]

Russ,

This explains ICMP and explains how to troubleshoot:

Using ICMP to troubleshoot TCP/IP networks

 

[TwoFace]

Russ, have you tried plugging another comp into the Linksys switch and see if it can comunicate with eth0 on the server? I'm not sure, but it sounds to me like something might be wrong with that config. If it works internally doesn't that only imply that eth1 is working correctly? Keep in mind these are only suggestions, I'm no networking expert by a long shot!

Here's what I'd try to do: First I'd assign only one IP to eth0, then use another computer on the same switch with one of the other IPs and see if they talk with eachother. Then try adding one virtual IP and see if everything still is A OK... You said all the IPs from 138 - 141 are virtual, shouldn't at least one be normal and not virtual?

These are just my thoughts at the moment, hopefully someone with real knowledge will help you out tho'!

Good luck

With love and respect your fellow TA member

Two-Face

 

[Drakkhen]

Well, the first thing that I would try would be to take off the port forwarding to the eth1 interface and try the traceroute again. If that works, the DSL router is just getting a little confused.

Now, when you say port forwarding, are you forwarding all of the ports to that one box? If so, then the reply from the 137 address is correct. The DSL router forwards the traceroute packet to the eth1 interface, and since it is the same box, it just replies saying that you have gotten to your destination.

But, in looking at your diagram, if you have 4 public addresses assigned to that one box, why are you port forwarding to the 137 address? Why not just use one of the 4 addresses that you already have exposed.

Good luck!

 

[Kilowatt]

Man what a Clusterf*ck layout you have there. :Q and it's actualy working?!?j/k

Does the Cisco only have one port?
I beleive I'd remove the switch between the Cisco and the Linksys, and plug Eth0 directy into the Cisco if possible.
If not, I'd plug it into the Linksys, but either way, I'd remove the switch between the Cisco and the Linksys.
Maybe, if the Cisco only has one port, I'd switch the Linksys and the Cisco posistions, and use the Cisco as the DHCP/DNS server for the rest of your machines.

Do the virtual IP's on Eth0 have content on those ports?
Are the ports different on each virtual domain?
Is your Apache server setup with nothing on the &quot;main domain&quot;, and all other virtual domains below the main.

/me is just shooting in the dark

 

[Captain_Rob]

I think I understand why you want dual NIC's. With Dual NICS, you can bind various services to a specific NIC. This allows you to setup services on your internal LAN &quot;for internal use only&quot; (like telnet and samba). Although, I have to say you sure are jumping head first into networking with a setup like that.

I'm assuming you have the NAT router set to FORWORD just for testing and to get it working for now, but that will need to be disabled. What does the route table look like on the server? I'm guessing it shows a default route pointing to NAT router (0.0.0.0 0.0.0.0 192.168.1.1). Your route table should not show anything pointing to your NAT router (&quot;man route&quot; for help). You must manually set the IP of eth1, with no gateway.

Make sure you removed the eth1 IP from the DHCP scope on the NAT router, and it is in GATEWAY mode.

The route table on the server should look something like this (loopback and broadcast routes removed for simplcity)

192.168.1.0 255.255.255.0 192.168.1.a
207.108.218.b 255.255.255.c 207.108.218.138
0.0.0.0 0.0.0.0 207.108.218.d

&quot;a&quot;=eth1 / &quot;b&quot;=network ID, which is based on your mask of &quot;c&quot; / &quot;d&quot;=675 internal IP, AKA your gateway. (I'm guessing c=248, thus b=136 / Make sure you are setting the mask correctly on everything dealing with the 207.108.218.x network!)

If the above is all correct, and outside access to eth0 still doesn't work, you could have a 675 configuration issue, or an ISP issue further upstream.

Do you have admin rights on the 675? I'm not fimiliar with the 675, but if it's similar to other Cisco gear I might be able to help with that if needed.

GOOD LUCK!

[Edit]

I also think you need set the NAT router to filter the eth1 address, but to do that you can't use DHCP. This would mean you'd have to setup a DHCP server on eth1 for your crackrack.

Oh, I forgot to mention I'm home sick today and on some good drugs, so I might not be think clearly

[Edit]

 

[bphantom]

Captain_Rob is correct. It is a routing issue. I ran into a very similiar problem when trying to multihome two connections to my firewall (cable modem and DSL). The standard routing of the Linux 2.2 kernels is (I believe 2.4 uses the more advanced routing), well dumb. Whatever is your default gateway, will be the path any outgoing responses will head out of. So if eth1 is your default gateway, all pings heading towards eth0 aliases with go out eth1. There is a package called Gated that replaces the default routing package, but I didn't have the time to mess with it and went a separate way (two firewalls...).

It's really messed up when pinging my DSL IP and it's trying to send the reply out my cable modem (default route). The reply would get killed by the ipchains ruleset and drove me insane trying to figure out what happened to the reply! D'oh!

Brad..

 

[Russ]

Man, I love you guys. So much info to digest.

divide_by_zero,

Thanks, I'll sink my teeth in to that info later today.

TwoFace,

Yes, I've verified that all the hardware is functioning correctly by trying different combinations. All communications function in all directions, with the exception of the additional IP addresses.

On Eth0, in addition to the assigment of the four WAN addresses, there is also a fixed &quot;non&quot; virtual LAN address assigned.

Drakkhen,

I've tried shutting down port forwarding on 137, and then nothing will ping at all. The reason I currently have it enabled is it's the only way I can currently test my Apache setup from outside the network. Inside the network everything works like a champ. All the IP's load up just as they should, but that's because they're only one hop away.

Kilowatt,

I tried plugging the Cisco directly in to Eth0 on the server. Nothing on the entire network functions when I do that. I also would like to have it setup so that web traffic does not go thru the Linksys DSL router at all, but is directed ONLY to Eth0. This gives me addtional security.

Captain_Rob,

Your assumptions about the reasons for two NICs and the port forwarding are right on the money. By assigning only a LAN address to Eth1, I can bind the potentially problematic services to only that card and reduce the possibility of compromise.

Once I have the addresses assigned to Eth0 working properly, I will shut down the port forwarding to Eth1 so that the only access to it will be internal.

The Eth1 IP is already removed from the NAT router (the Linksys Router), and the router is in Gateway mode. The routing table is correct with the correct network identifier, mask and gateway for the WAN, and a seperate mask and gateway for the LAN.



<< 192.168.1.0 255.255.255.0 192.168.1.a
207.108.218.b 255.255.255.c 207.108.218.138
0.0.0.0 0.0.0.0 207.108.218.d

&quot;a&quot;=eth1 / &quot;b&quot;=network ID, which is based on your mask of &quot;c&quot; / &quot;d&quot;=675 internal IP, AKA your gateway. (I'm guessing c=248, thus b=136 / Make sure you are setting the mask correctly on everything dealing with the 207.108.218.x network!) >>



Again, your assumptions are correct. You know your stuff! Yes, the mask is set correctly for the WAN addresses, and yes it is .248. But, &quot;b&quot; is not 136, it is 143 which is the broadcast address. 136 is the network identifier. Should it not be the broadcast?

If by admin rights you mean can I get in there and change anything? Sure. I've been in there so many times now changing settings and re-writing the damned thing that when I close my eyes I see the CBOS command line.

Thanks again, guys!

Russ, NCNE

 

[Lord Demios]

I can't wait to have to deal with more of this fun stuff, when GB goes to many machines. Routing is fun, but when it doesn't work it is a pain in the butt. This is also why Brad lives pretty close, and I have his number on speed dial.

/me can't wait for the routing fun to begin when we move the colocation's to the new building. oh yea, that will be fun!

LD

 

[Captain_Rob]

<< Should it not be the broadcast? >>



Actually you should have both the Network ID and the Broadcast ID listed in your route tables.

Network ID / Mask / Gateway or Next Hop / Interface

0.0.0.0 0.0.0.0 207.108.218.142 207.108.218.138
207.108.218.136 255.255.255.248 207.108.218.142 207.108.218.138
207.108.218.143 255.255.255.255 207.108.218.142 207.108.218.138




<< On Eth1, in addition to the assigment of the four WAN addresses, there is also a fixed &quot;non&quot; virtual LAN address assigned >>


(assuming you meant Eth0) What is the IP actually assigned to eth0?


<< 138, 139, 140, and 141. These are all assigned as virtual IP's to Eth0 >>


The only usable IP's on that subnet are 137-142, and all your other IP's are accounted for.


{q]I tried plugging the Cisco directly in to Eth0 on the server. Nothing on the entire network functions when I do that[/i] >>



Could you ping the &quot;fixed&quot; address from the 675, and the 675 from eth0? If yes, then this again this points to a routing problem on either the 675 or eth0.

If you want, post or email me your full route table, the interface setups (linux version of winipcfg), and the 675 config might also be helpfull, including it's route table.

If you can get by without the extra virtual IP's, you might consider changing your layout a little. You could put both eth0 and eth1 on your inside LAN. The NAT router could forward the outside world to eth1, and eth0 would be your address which your protected services run on.

 

[Russ]

Captain_Rob,



<< Actually you should have both the Network ID and the Broadcast ID listed in your route tables.

Network ID / Mask / Gateway or Next Hop / Interface >>



Yep, they do both appear in the tables for the router. My screw up as I thought you were referring to the Linux ifconfig. In there, just the IP/mask/broadcast appear.



<< (assuming you meant Eth0) What is the IP actually assigned to eth0? >>



Yep, that's what I meant. The actual IP assigned is a LAN address of 192.168.1.23, mask of 255.255.255.0 and broadcast of 192.168.1.255.



<< The only usable IP's on that subnet are 137-142, and all your other IP's are accounted for. >>



Yep, I know. 137 thru 141 are assignable by me. 136 is the identifier, 142 the gateway and 143 is the broadcast.



<< Could you ping the &quot;fixed&quot; address from the 675, and the 675 from eth0? If yes, then this again this points to a routing problem on either the 675 or eth0. >>



Nope. No communications, including pings.



<< If you want, post or email me your full route table, the interface setups (linux version of winipcfg), and the 675 config might also be helpfull, including it's route table. >>



HeHe. I'll try to figure out how to copy and paste the ifconfig from Linux. The 675 config is easy; I've printed that puppy out quite a few times over the last couple weeks.



<< If you can get by without the extra virtual IP's, you might consider changing your layout a little. You could put both eth0 and eth1 on your inside LAN. The NAT router could forward the outside world to eth1, and eth0 would be your address which your protected services run on. >>



Nope, I need them, which is why I leased them. The goal of this experiment is to move my low traffic sites to my own server so I can stop paying hosting fees. This is also why I have a backup server in the mix.

Now, everything that I've read and researched over the past two weeks, and the input I've gotten from all you guys that know much more about this then I do, points to the configuration of the Cisco 675.

Something in it is preventing the requests to the four WAN IP's (138-141) from getting through, and instead directing the traffic to the Linksys router at 137. The frustrating part is that I've been unable to figure out what that is. I want to set it up so that the Cisco is just a pipe, a point of entry that allows the traffic to move to the requested destination.

There is some feature and or service/s the Cisco is providing that is clogging the pipe.

Thanks again!!!!

Russ, NCNE

 

[Captain_Rob]

<< The actual IP assigned is a LAN address of 192.168.1.23 >>



etho must have a valid IP address in the 207.108.218.x range if it is physically attached to that network.

 

[Russ]

Captain_Rob,

It does. It has four of them (138-141) setup as aliases (virtual) that point to the LAN address. Each of these is assigned the WAN mask, gateway and broadcast. Do you mean that the ACTUAL, not virtual, address also has to be within the WAN range? And the attendant mask, gateway and broadcast?

If so, will that interfere with the other addresses that are aliased? For example, if I assign 207.108.218.138 physically to Eth0, will it have any problems with 139, 140 and 141 assigned as virtual?

Russ, NCNE

 

[Russ]

Okay, I removed the internal LAN address, and all the virtual addresses from Eth0 and assigned 207.108.218.138 with 255.255.255.248 as the mask, 207.108.218.142 as the gateway and 207.108.218.143 as the broadcast. I can load it internally just like always, but externally it still pings to 137.

I think this has got to be something in the Cisco. Maybe I'll go do something pleasant for a while, like getting a root canal, and return to this later.

Russ, NCNE

 

[Kilowatt]

LOL Yeah, step away for the computer.

Maybe go and assimalate Janet Reno's box. :Q <hehe>

 

[Captain_Rob]

Did you disable forwarding on the Linksys?

I'm starting to think this is a routing issue with the 675. Did you ever look at it's route table? It should look something like:

0.0.0.0 0.0.0.0 207.224.249.102 207.224.249.101
207.224.249.100 255.255.255.252 207.224.249.101 207.224.249.101
207.108.218.136 255.255.255.248 207.108.218.142 207.108.218.142

or in normal Cisco language

207.224.249.0 is subnetted, 1 subnets
C 207.224.249.100/30 is directly connected, {DSL Interface}
207.108.218.0 is subnetted, 1 subnets
C 207.108.218.136/29 is directly connected, {LAN Interface}
S* 0.0.0.0/0 [1/0] via 207.224.249.101


On a long shot though, what does the route table look like on the Linksys? (advanced, static route, show routing table) It should show something like:

0.0.0.0 0.0.0.0 207.108.218.142 # WAN
207.108.218.136 255.255.255.248 207.108.218.142 # WAN
192.168.1.0 255.255.255.0 192.168.1.1 # LAN


The Linksys is capable of doing Dynamic Routing, but I really don't think you need to do this since as far as the ISP is concerned you only have the 1 subnet they are providing and routing. The 192.168.x.x subnet is a public range and hidden by NAT. It's easy to try though. I'd assume the 675 would be compatible with RIP-2.

I'm starting to see double now, no wonder I'm sick :frown:

 

[Russ]

<< Did you disable forwarding on the Linksys? >>



Yep. When I do this, 138 thru 141 still ping to 137, and none will load.



<< I'm starting to think this is a routing issue with the 675. Did you ever look at it's route table? >>



This is the route table from the Cisco 675:

cbos#show route
[TARGET] [MASK] [GATEWAY] [M] [TYPE] [IF] [AGE]
0.0.0.0 0.0.0.0 0.0.0.0 1 SA WAN0-0 0
207.108.218.136 255.255.255.248 0.0.0.0 1 LA ETH0 0
63.0.0.0 255.0.0.0 0.0.0.0 1 A WAN0-0 0
63.226.196.254 255.255.255.255 0.0.0.0 1 AH WAN0-0 0



<< On a long shot though, what does the route table look like on the Linksys? >>



It looks like this, in both the dynamic and the static tables:

Destination LAN IP Subnet Mask Default Gateway Hop Count Interface
0.0.0.0 0.0.0.0 207.108.218.142 1 WAN
192.168.1.0 255.255.255 .0 0.0.0.0 1 LAN
207.108.218.136 255.255.255.248 0.0.0.0 1 WAN

Russ, NCNE

 

[Russ]

YEEEHHAAAWWWW! Thanks to Captain_Rob, the problem is solved and all of my IP addresses are working! He spotted it over in the networking forum when I posted the routing table from my server.

Default route was set to 192.168.1.1 at Eth1. Changed that to 207.108.218.142 at Eth0, reboot, and BINGO! It works!

Russ, NCNE

 

[Kilowatt]

Well that's cool.

But now 207.108.218.137 doesn't work, but the others do.

I'm just wondering, are all of the IPs pointing to the same http dir, or are they all copies of the same thing in different dirs?

 

[Russ]

Kilowatt,

That's because I shut down port forwarding on 137. The others were all pointing to the same directory. Only 138 is actually setup as a virtual host right now, but the default VH was also the same directory (TA Cube).

I now have only 138 pointing to the tacube directory, and the default virtual server (catches 139-141) pointed to a different directory.

Russ, NCNE