Tuesday, April 29, 2014

Hadoop Ecosystem Default Port

 1. HDFS Ports

The following table lists the default ports used by the various HDFS services.
Table 2.1. HDFS Ports
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
NameNode WebUI
Master Nodes (NameNode and any back-up NameNodes)50070httpWeb UI to look at current status of HDFS, explore file systemYes (Typically admins, Dev/Support teams)dfs.http.address
50470httpsSecure http servicedfs.https.address
NameNode metadata service
Master Nodes (NameNode and any back-up NameNodes)8020/9000IPC
File system metadata operations
Yes (All clients who directly need to interact with the HDFS)Embedded in URI specified by fs.default.name
DataNode
All Slave Nodes
50075
http
DataNode WebUI to access the status, logs etc.
Yes (Typically admins, Dev/Support teams)dfs.datanode.http.address
50475
https
Secure http service
dfs.datanode.https.address
50010
Data transfer
dfs.datanode.address
50020
IPC
Metadata operations
Nodfs.datanode.ipc.address
Secondary NameNodeSecondary NameNode and any backup Secondanry NameNode
50090
http
Checkpoint for NameNode metadata
Nodfs.secondary.http.address

2. MapReduce Ports

The following table lists the default ports used by the various MapReduce services.
Table 2.2. MapReduce Ports
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
JobTracker  WebUI
Master Nodes (JobTracker Node and any back-up Job­Tracker node )50030httpWeb UI for JobTrackerYesmapred.job.tracker.http.address
JobTracker
Master Nodes (JobTracker Node)8021IPC
For job submissions
Yes (All clients who need to submit the MapReduce jobs  including Hive, Hive server, Pig)Embedded in URI specified bymapred.job.tracker
Task­Tracker Web UI and Shuffle
All Slave Nodes
50060
httpDataNode Web UI to access status, logs, etc.Yes (Typically admins, Dev/Support teams)mapred.task.tracker.http.address
History Server WebUI51111httpWeb UI for Job HistoryYesmapreduce.history.server.http.address

 3. Hive Ports

The following table lists the default ports used by the various Hive services.
[Note]Note
Neither of these services are used in a standard HDP installation.
Table 2.3. Hive Ports
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
Hive Server2
Hive Server machine (Usually a utility machine)10000thriftService for programatically (Thrift/JDBC) connecting to HiveYes (Clients who need to connect to Hive either programatically or through UI SQL tools that use JDBC)ENV Variable HIVE_PORT
Hive Metastore
9083thriftYes (Clients that run Hive, Pig and potentially M/R jobs that use HCatalog)hive.metastore.uris

4. HBase Ports

The following table lists the default ports used by the various HBase services.
Table 2.4. HBase Ports
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
HMaster
Master Nodes (HBase Master Node and any back-up HBase Master node)60000Yeshbase.master.port
HMaster Info Web UI
Master Nodes (HBase master Node and back up HBase Master node if any)60010httpThe port for the HBase­Master web UI. Set to -1 if you do not want the info server to run.Yeshbase.master.info.port
Region Server
All Slave Nodes60020Yes (Typically admins, dev/support teams)hbase.regionserver.port
Region Server
All Slave Nodes60030httpYes (Typically admins, dev/support teams)hbase.regionserver.info.port
All ZooKeeper Nodes2888Port used by ZooKeeper peers to talk to each other.Seehere for more information.Nohbase.zookeeper.peerport
All ZooKeeper Nodes3888Port used by ZooKeeper peers to talk to each other.Seehere for more information.hbase.zookeeper.leaderport
2181Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect.hbase.zookeeper.property.clientPort

5. WebHCat Port

The following table lists the default ports used by the WebHCat service.
Table 2.5. WebHCat Port
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
WebHCat Server
Any utility machine50111httpWeb API on top of HCatalog and other Hadoop servicesYestempleton.port

 6. Ganglia Ports

The following table lists the default ports used by the various Ganglia services.

Table 2.6. Ganglia Ports
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
Ganglia server8660/61/62/63For gmond collectors
All Slave Nodes8660For gmond agents
Ganglia server8651For ganglia gmetad

 7. MySQL Ports

The following table lists the default ports used by the various MySQL services.
Table 2.7. MySQL Ports
ServiceServersDefault Ports UsedProtocolDescriptionNeed End User Access?Configuration Parameters
MySQLMySQL database server3306

No comments:

Post a Comment