aboutsummaryrefslogtreecommitdiff
path: root/aarch64/share/doc/hadoop/hdfs/CHANGES.txt
diff options
context:
space:
mode:
Diffstat (limited to 'aarch64/share/doc/hadoop/hdfs/CHANGES.txt')
-rw-r--r--aarch64/share/doc/hadoop/hdfs/CHANGES.txt6945
1 files changed, 6945 insertions, 0 deletions
diff --git a/aarch64/share/doc/hadoop/hdfs/CHANGES.txt b/aarch64/share/doc/hadoop/hdfs/CHANGES.txt
new file mode 100644
index 0000000..6c385fb
--- /dev/null
+++ b/aarch64/share/doc/hadoop/hdfs/CHANGES.txt
@@ -0,0 +1,6945 @@
+Hadoop HDFS Change Log
+
+Release 2.2.0 - 2013-10-13
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ HDFS-5230. Introduce RpcInfo to decouple XDR classes from the RPC API.
+ (Haohui Mai via brandonli)
+
+ IMPROVEMENTS
+
+ HDFS-5246. Make Hadoop nfs server port and mount daemon port
+ configurable. (Jinghui Wang via brandonli)
+
+ HDFS-5256. Use guava LoadingCache to implement DFSClientCache. (Haohui Mai
+ via brandonli)
+
+ HDFS-5308. Replace HttpConfig#getSchemePrefix with implicit schemes in HDFS
+ JSP. (Haohui Mai via jing9)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-5139. Remove redundant -R option from setrep.
+
+ HDFS-5251. Race between the initialization of NameNode and the http
+ server. (Haohui Mai via suresh)
+
+ HDFS-5258. Skip tests in TestHDFSCLI that are not applicable on Windows.
+ (Chuan Liu via cnauroth)
+
+ HDFS-5186. TestFileJournalManager fails on Windows due to file handle leaks.
+ (Chuan Liu via cnauroth)
+
+ HDFS-5031. BlockScanner scans the block multiple times. (Vinay via Arpit
+ Agarwal)
+
+ HDFS-5268. NFS write commit verifier is not set in a few places (brandonli)
+
+ HDFS-5265. Namenode fails to start when dfs.https.port is unspecified.
+ (Haohui Mai via jing9)
+
+ HDFS-5255. Distcp job fails with hsftp when https is enabled in insecure
+ cluster. (Arpit Agarwal)
+
+ HDFS-5279. Guard against NullPointerException in NameNode JSP pages before
+ initialization of FSNamesystem. (cnauroth)
+
+ HDFS-5289. Race condition in TestRetryCacheWithHA#testCreateSymlink causes
+ spurious test failure. (atm)
+
+ HDFS-5300. FSNameSystem#deleteSnapshot() should not check owner in case of
+ permissions disabled. (Vinay via jing9)
+
+ HDFS-5306. Datanode https port is not available at the namenode. (Suresh
+ Srinivas via brandonli)
+
+ HDFS-5299. DFS client hangs in updatePipeline RPC when failover happened.
+ (Vinay via jing9)
+
+ HDFS-5259. Support client which combines appended data with old data
+ before sends it to NFS server. (brandonli)
+
+Release 2.1.1-beta - 2013-09-23
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ HDFS-4962 Use enum for nfs constants (Nicholas SZE via jing9)
+
+ HDFS-5071 Change hdfs-nfs parent project to hadoop-project (brandonli)
+
+ HDFS-4763 Add script changes/utility for starting NFS gateway (brandonli)
+
+ HDFS-5076 Add MXBean methods to query NN's transaction information and
+ JournalNode's journal status. (jing9)
+
+ HDFS-5104 Support dotdot name in NFS LOOKUP operation (brandonli)
+
+ HDFS-5107 Fix array copy error in Readdir and Readdirplus responses
+ (brandonli)
+
+ HDFS-5110 Change FSDataOutputStream to HdfsDataOutputStream for opened
+ streams to fix type cast error. (brandonli)
+
+ HDFS-5069 Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for
+ NFS deployment (brandonli)
+
+ HDFS-4947 Add NFS server export table to control export by hostname or
+ IP range (Jing Zhao via brandonli)
+
+ HDFS-5078 Support file append in NFSv3 gateway to enable data streaming
+ to HDFS (brandonli)
+
+ HDFS-5136 MNT EXPORT should give the full group list which can mount the
+ exports (brandonli)
+
+ HDFS-5118. Provide testing support for DFSClient to drop RPC responses.
+ (jing9)
+
+ HDFS-5085. Refactor o.a.h.nfs to support different types of
+ authentications. (jing9)
+
+ HDFS-5067 Support symlink operations in NFS gateway. (brandonli)
+
+ HDFS-5199 Add more debug trace for NFS READ and WRITE. (brandonli)
+
+ HDFS-5234 Move RpcFrameDecoder out of the public API.
+ (Haohui Mai via brandonli)
+
+ IMPROVEMENTS
+
+ HDFS-4513. Clarify in the WebHDFS REST API that all JSON respsonses may
+ contain additional properties. (szetszwo)
+
+ HDFS-5061. Make FSNameSystem#auditLoggers an unmodifiable list.
+ (Arpit Agarwal via suresh)
+
+ HDFS-4905. Add appendToFile command to "hdfs dfs". (Arpit Agarwal via
+ cnauroth)
+
+ HDFS-4926. Namenode webserver's page has a tooltip that is inconsistent
+ with the datanode HTML link. (Vivek Ganesan via jing9)
+
+ HDFS-5047. Supress logging of full stack trace of quota and lease
+ exceptions. (Robert Parker via kihwal)
+
+ HDFS-5111. Remove duplicated error message for snapshot commands when
+ processing invalid arguments. (jing9)
+
+ HDFS-5045. Add more unit tests for retry cache to cover all AtMostOnce
+ methods. (jing9)
+
+ HDFS-3245. Add metrics and web UI for cluster version summary. (Ravi
+ Prakash via kihwal)
+
+ HDFS-5128. Allow multiple net interfaces to be used with HA namenode RPC
+ server. (kihwal)
+
+ HDFS-5150. Allow per NN SPN for internal SPNEGO. (kihwal)
+
+ HDFS-4680. Audit logging of delegation tokens for MR tracing. (Andrew Wang)
+
+ HDFS-5212. Refactor RpcMessage and NFS3Response to support different
+ types of authentication information. (jing9)
+
+ HDFS-4971. Move IO operations out of locking in OpenFileCtx. (brandonli and
+ jing9)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-5043. For HdfsFileStatus, set default value of childrenNum to -1
+ instead of 0 to avoid confusing applications. (brandonli)
+
+ HDFS-5028. LeaseRenewer throws ConcurrentModificationException when timeout.
+ (zhaoyunjiong via szetszwo)
+
+ HDFS-4993. Fsck can fail if a file is renamed or deleted. (Robert Parker
+ via kihwal)
+
+ HDFS-5091. Support for spnego keytab separate from the JournalNode keytab
+ for secure HA. (jing9)
+
+ HDFS-5051. nn fails to download checkpointed image from snn in some
+ setups. (Vinay and suresh via suresh)
+
+ HDFS-4898. BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to
+ properly fallback to local rack. (szetszwo)
+
+ HDFS-4632. globStatus using backslash for escaping does not work on Windows.
+ (Chuan Liu via cnauroth)
+
+ HDFS-5080. BootstrapStandby not working with QJM when the existing NN is
+ active. (jing9)
+
+ HDFS-5099. Namenode#copyEditLogSegmentsToSharedDir should close
+ EditLogInputStreams upon finishing. (Chuan Liu via cnauroth)
+
+ HDFS-2994. If lease soft limit is recovered successfully
+ the append can fail. (Tao Luo via shv)
+
+ HDFS-5100. TestNamenodeRetryCache fails on Windows due to incorrect cleanup.
+ (Chuan Liu via cnauroth)
+
+ HDFS-5103. TestDirectoryScanner fails on Windows. (Chuan Liu via cnauroth)
+
+ HDFS-5102. Snapshot names should not be allowed to contain slash characters.
+ (jing9)
+
+ HDFS-5105. TestFsck fails on Windows. (Chuan Liu via arp)
+
+ HDFS-5106. TestDatanodeBlockScanner fails on Windows due to incorrect path
+ format. (Chuan Liu via cnauroth)
+
+ HDFS-4594. WebHDFS open sets Content-Length header to what is specified by
+ length parameter rather than how much data is actually returned. (cnauroth)
+
+ HDFS-5124. DelegationTokenSecretManager#retrievePassword can cause deadlock
+ in NameNode. (Daryn Sharp via jing9)
+
+ HDFS-5132. Deadlock in NameNode between SafeModeMonitor#run and
+ DatanodeManager#handleHeartbeat. (kihwal)
+
+ HDFS-5077. NPE in FSNamesystem.commitBlockSynchronization().
+ (Plamen Jeliazkov via shv)
+
+ HDFS-5140. Too many safemode monitor threads being created in the standby
+ namenode causing it to fail with out of memory error. (jing9)
+
+ HDFS-5159. Secondary NameNode fails to checkpoint if error occurs
+ downloading edits on first checkpoint. (atm)
+
+ HDFS-5192. NameNode may fail to start when
+ dfs.client.test.drop.namenode.response.number is set. (jing9)
+
+ HDFS-5219. Add configuration keys for retry policy in WebHDFSFileSystem.
+ (Haohui Mai via jing9)
+
+ HDFS-5231. Fix broken links in the document of HDFS Federation. (Haohui Mai
+ via jing9)
+
+ HDFS-5249. Fix dumper thread which may die silently. (brandonli)
+
+Release 2.1.0-beta - 2013-08-22
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-4053. Increase the default block size. (eli)
+
+ HDFS-4305. Add a configurable limit on number of blocks per file, and min
+ block size. (Andrew Wang via atm)
+
+ HDFS-4434. Provide a mapping from INodeId to INode. (suresh)
+
+ HDFS-2802. Add HDFS Snapshot feature. (See breakdown of tasks below for
+ subtasks and contributors)
+
+ HDFS-4866. Protocol buffer support cannot compile under C. (Arpit Agarwal via
+ cnauroth)
+
+ HDFS-5083. Update the HDFS compatibility version range. (kihwal)
+
+ NEW FEATURES
+
+ HDFS-1804. Add a new block-volume device choosing policy that looks at
+ free space. (atm)
+
+ HDFS-4296. Reserve layout version for release 1.2.0. (suresh)
+
+ HDFS-4334. Add a unique id to INode. (Brandon Li via szetszwo)
+
+ HDFS-4339. Persist inode id in fsimage and editlog. (Brandon Li via
+ suresh)
+
+ HDFS-4340. Update addBlock() to inculde inode id as additional argument.
+ (Brandon Li via suresh)
+
+ HDFS-4502. JsonUtil.toFileStatus(..) should check if the fileId property
+ exists. (Brandon Li via suresh)
+
+ HDFS-2576. Enhances the DistributedFileSystem's create API so that clients
+ can specify favored datanodes for a file's blocks. (ddas)
+
+ HDFS-347. DFS read performance suboptimal when client co-located on nodes
+ with data. (Colin Patrick McCabe via todd and atm)
+
+ HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
+ Azure environments. (See breakdown of tasks below for subtasks and
+ contributors)
+
+ HDFS-3601. Add BlockPlacementPolicyWithNodeGroup to support block placement
+ with 4-layer network topology. (Junping Du via szetszwo)
+
+ HDFS-3495. Update Balancer to support new NetworkTopology with NodeGroup.
+ (Junping Du via szetszwo)
+
+ HDFS-4659 Support setting execution bit for regular files (Brandon Li via sanjay)
+
+ HDFS-4762 Provide HDFS based NFSv3 and Mountd implementation (brandonli)
+
+ HDFS-4372. Track NameNode startup progress. (cnauroth)
+
+ HDFS-4373. Add HTTP API for querying NameNode startup progress. (cnauroth)
+
+ HDFS-4374. Display NameNode startup progress in UI. (cnauroth)
+
+ HDFS-4974. Add Idempotent and AtMostOnce annotations to namenode
+ protocol methods. (suresh)
+
+ HDFS-4979. Implement retry cache on Namenode. (suresh)
+
+ HDFS-5025. Record ClientId and CallId in EditLog to enable rebuilding
+ retry cache in case of HA failover. (Jing Zhao via suresh)
+
+ IMPROVEMENTS
+
+ HDFS-4461. DirectoryScanner: volume path prefix takes up memory for every
+ block that is scanned (Colin Patrick McCabe)
+
+ HDFS-4222. NN is unresponsive and loses heartbeats from DNs when
+ configured to use LDAP and LDAP has issues. (Xiaobo Peng, suresh)
+
+ HDFS-4304. Make FSEditLogOp.MAX_OP_SIZE configurable. (Colin Patrick
+ McCabe via atm)
+
+ HDFS-4518. Finer grained metrics for HDFS capacity.
+ (Arpit Agarwal via suresh)
+
+ HDFS-4519. Support overriding jsvc binary and log file locations
+ when launching secure datanode. (Chris Nauroth via suresh)
+
+ HDFS-4569. Small image transfer related cleanups.
+ (Andrew Wang via suresh)
+
+ HDFS-4521. Invalid network toploogies should not be cached. (Colin Patrick
+ McCabe via atm)
+
+ HDFS-4246. The exclude node list should be more forgiving, for each output
+ stream. (harsh via atm)
+
+ HDFS-4635. Move BlockManager#computeCapacity to LightWeightGSet. (suresh)
+
+ HDFS-4621. Additional logging to help diagnose slow QJM syncs. (todd)
+
+ HDFS-4618. Default transaction interval for checkpoints is too low. (todd)
+
+ HDFS-4525. Provide an API for knowing that whether file is closed or not.
+ (SreeHari via umamahesh)
+
+ HDFS-3940. Add Gset#clear method and clear the block map when namenode is
+ shutdown. (suresh)
+
+ HDFS-4679. Namenode operation checks should be done in a consistent
+ manner. (suresh)
+
+ HDFS-4693. Some test cases in TestCheckpoint do not clean up after
+ themselves. (Arpit Agarwal, suresh via suresh)
+
+ HDFS-3817. Avoid printing SafeModeException stack trace.
+ (Brandon Li via suresh)
+
+ HDFS-4124. Refactor INodeDirectory#getExistingPathINodes() to enable
+ returning more than INode array. (Jing Zhao via suresh)
+
+ HDFS-4151. Change the methods in FSDirectory to pass INodesInPath instead
+ of INode[] as a parameter. (szetszwo)
+
+ HDFS-4129. Add utility methods to dump NameNode in memory tree for
+ testing. (szetszwo via suresh)
+
+ HDFS-4152. Add a new class BlocksMapUpdateInfo for the parameter in
+ INode.collectSubtreeBlocksAndClear(..). (Jing Zhao via szetszwo)
+
+ HDFS-4206. Change the fields in INode and its subclasses to private.
+ (szetszwo)
+
+ HDFS-4215. Remove locking from addToParent(..) since it is used in image
+ loading, and add INode.isFile(). (szetszwo)
+
+ HDFS-4243. When replacing an INodeDirectory, the parent pointers of the
+ children of the child have to be updated to the new child. (Jing Zhao
+ via szetszwo)
+
+ HDFS-4209. Clean up the addNode/addChild/addChildNoQuotaCheck methods in
+ FSDirectory and INodeDirectory. (szetszwo)
+
+ HDFS-4346. Add SequentialNumber as a base class for INodeId and
+ GenerationStamp. (szetszwo)
+
+ HDFS-4721. Speed up lease recovery by avoiding stale datanodes and choosing
+ the datanode with the most recent heartbeat as the primary. (Varun Sharma
+ via szetszwo)
+
+ HDFS-4804. WARN when users set the block balanced preference percent below
+ 0.5 or above 1.0. (Stephen Chu via atm)
+
+ HDFS-4698. Provide client-side metrics for remote reads, local reads, and
+ short-circuit reads. (Colin Patrick McCabe via atm)
+
+ HDFS-3498. Support replica removal in BlockPlacementPolicy and make
+ BlockPlacementPolicyDefault extensible for reusing code in subclasses.
+ (Junping Du via szetszwo)
+
+ HDFS-4234. Use generic code for choosing datanode in Balancer. (szetszwo)
+
+ HDFS-4880. Print the image and edits file loaded by the namenode in the
+ logs. (Arpit Agarwal via suresh)
+
+ HDFS-2572. Remove unnecessary double-check in DN#getHostName. (harsh)
+
+ HDFS-2857. Cleanup BlockInfo class. (suresh)
+
+ HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using
+ NetUtils. (Hari Mankude via suresh)
+
+ HDFS-4914. Use DFSClient.Conf instead of Configuration. (szetszwo)
+
+ HDFS-4883. complete() should verify fileId. (Tao Luo via shv)
+
+ HDFS-4772. Add number of children in HdfsFileStatus. (brandonli)
+
+ HDFS-4932. Avoid a wide line on the name node webUI if we have more Journal
+ nodes. (Fengdong Yu via cnauroth)
+
+ HDFS-4908. Reduce snapshot inode memory usage. (szetszwo)
+
+ HDFS-4645. Move from randomly generated block ID to sequentially generated
+ block ID. (Arpit Agarwal via szetszwo)
+
+ HDFS-4912. Cleanup FSNamesystem#startFileInternal. (suresh)
+
+ HDFS-4903. Print trash configuration and trash emptier state in
+ namenode log. (Arpit Agarwal via suresh)
+
+ HDFS-4992. Make balancer's mover thread count and dispatcher thread count
+ configurable. (Max Lapan via szetszwo)
+
+ HDFS-4996. ClientProtocol#metaSave can be made idempotent by overwriting the
+ output file instead of appending to it. (cnauroth)
+
+ HADOOP-9418. Add symlink support to DistributedFileSystem (Andrew Wang via
+ Colin Patrick McCabe)
+
+ HDFS-5007. Replace hard-coded property keys with DFSConfigKeys fields.
+ (Kousuke Saruta via jing9)
+
+ HDFS-5008. Make ClientProtocol#abandonBlock() idempotent. (jing9)
+
+ HADOOP-9760. Move GSet and related classes to common from HDFS.
+ (suresh)
+
+ HDFS-5020. Make DatanodeProtocol#blockReceivedAndDeleted idempotent.
+ (jing9)
+
+ HDFS-5024. Make DatanodeProtocol#commitBlockSynchronization idempotent.
+ (Arpit Agarwal via jing9)
+
+ HDFS-3880. Use Builder to build RPC server in HDFS.
+ (Brandon Li and Junping Du via szetszwo)
+
+ OPTIMIZATIONS
+
+ HDFS-4465. Optimize datanode ReplicasMap and ReplicaInfo. (atm)
+
+ HDFS-5027. On startup, DN should scan volumes in parallel. (atm)
+
+ BUG FIXES
+
+ HDFS-4626. ClientProtocol#getLinkTarget should throw an exception for
+ non-symlink and non-existent paths. (Andrew Wang via cmccabe)
+
+ HDFS-3934. duplicative dfs_hosts entries handled wrong. (Colin Patrick
+ McCabe)
+
+ HDFS-4470. Several HDFS tests attempt file operations on invalid HDFS
+ paths when running on Windows. (Chris Nauroth via suresh)
+
+ HDFS-4471. Namenode WebUI file browsing does not work with wildcard
+ addresses configured. (Andrew Wang via atm)
+
+ HDFS-4342. Directories configured in dfs.namenode.edits.dir.required
+ but not in dfs.namenode.edits.dir are silently ignored. (Arpit Agarwal
+ via szetszwo)
+
+ HDFS-4482. ReplicationMonitor thread can exit with NPE due to the race
+ between delete and replication of same file. (umamahesh)
+
+ HDFS-4269. Datanode rejects all datanode registrations from localhost
+ in single-node developer setup on Windows. (Chris Nauroth via suresh)
+
+ HDFS-4235. When outputting XML, OfflineEditsViewer can't handle some edits
+ containing non-ASCII strings. (Colin Patrick McCabe via atm)
+
+ HDFS-4541. Set hadoop.log.dir and hadoop.id.str when starting secure
+ datanode to write the logs to right dir by default. (Arpit Gupta via
+ suresh)
+
+ HDFS-4540. Namenode http server should use the web authentication
+ keytab for spnego principal. (Arpit Gupta via suresh)
+
+ HDFS-4544. Error in deleting blocks should not do check disk, for
+ all types of errors. (Arpit Agarwal via suresh)
+
+ HDFS-4565. Use DFSUtil.getSpnegoKeytabKey() to get the spnego keytab key
+ in secondary namenode and namenode http server. (Arpit Gupta via suresh)
+
+ HDFS-4571. WebHDFS should not set the service hostname on the server side.
+ (tucu)
+
+ HDFS-4013. TestHftpURLTimeouts throws NPE. (Chao Shi via suresh)
+
+ HDFS-4592. Default values for access time precision are out of sync between
+ hdfs-default.xml and the code. (atm)
+
+ HDFS-4522. LightWeightGSet expects incrementing a volatile to be atomic.
+ (Colin Patrick McCabe via atm)
+
+ HDFS-4484. libwebhdfs compilation broken with gcc 4.6.2. (Colin Patrick
+ McCabe via atm)
+
+ HDFS-4595. When short circuit read is fails, DFSClient does not fallback
+ to regular reads. (suresh)
+
+ HDFS-4583. TestNodeCount fails. (Ivan Mitic via suresh)
+
+ HDFS-4591. HA clients can fail to fail over while Standby NN is performing
+ long checkpoint. (atm)
+
+ HDFS-3277. fail over to loading a different FSImage if the first one we
+ try to load is corrupt. (Colin Patrick McCabe and Andrew Wang via atm)
+
+ HDFS-4596. Shutting down namenode during checkpointing can lead to md5sum
+ error. (Andrew Wang via atm)
+
+ HDFS-4614. FSNamesystem#getContentSummary should use getPermissionChecker
+ helper method. (atm)
+
+ HDFS-4620. Documentation for dfs.namenode.rpc-address specifies wrong
+ format. (Sandy Ryza via atm)
+
+ HDFS-4609. TestAuditLogs should release log handles between tests.
+ (Ivan Mitic via szetszwo)
+
+ HDFS-4598. Fix the default value of ConcatSourcesParam and the WebHDFS doc.
+ (szetszwo)
+
+ HDFS-4655. DNA_FINALIZE is logged as being an unknown command by the DN
+ when received from the standby NN. (atm)
+
+ HDFS-4656. DN heartbeat loop can be briefly tight. (atm)
+
+ HDFS-4658. Standby NN will log that it has received a block report "after
+ becoming active" (atm)
+
+ HDFS-4646. createNNProxyWithClientProtocol ignores configured timeout
+ value (Jagane Sundar via cos)
+
+ HDFS-3981. Fix handling of FSN lock in getBlockLocations. (Xiaobo Peng
+ and todd via todd)
+
+ HDFS-4676. TestHDFSFileSystemContract should set MiniDFSCluster variable
+ to null to free up memory. (suresh)
+
+ HDFS-4669. TestBlockPoolManager fails using IBM java. (Tian Hong Wang via
+ suresh)
+
+ HDFS-4643. Fix flakiness in TestQuorumJournalManager. (todd)
+
+ HDFS-4639. startFileInternal() should not increment generation stamp.
+ (Plamen Jeliazkov via shv)
+
+ HDFS-4695. TestEditLog leaks open file handles between tests.
+ (Ivan Mitic via suresh)
+
+ HDFS-4737. JVM path embedded in fuse binaries. (Sean Mackrory via atm)
+
+ HDFS-4739. NN can miscalculate the number of extra edit log segments to
+ retain. (atm)
+
+ HDFS-4745. TestDataTransferKeepalive#testSlowReader has race condition that
+ causes sporadic failure. (Chris Nauroth via suresh)
+
+ HDFS-4768. File handle leak in datanode when a block pool is removed.
+ (Chris Nauroth via suresh)
+
+ HDFS-4748. MiniJournalCluster#restartJournalNode leaks resources, which
+ causes sporadic test failures. (Chris Nauroth via suresh)
+
+ HDFS-4733. Make HttpFS username pattern configurable. (tucu via atm)
+
+ HDFS-4778. Fixes some issues that the first patch on HDFS-2576 missed.
+ (ddas)
+
+ HDFS-4785. Concat operation does not remove concatenated files from
+ InodeMap. (suresh)
+
+ HDFS-4784. NPE in FSDirectory.resolvePath(). (Brandon Li via suresh)
+
+ HDFS-4810. several HDFS HA tests have timeouts that are too short. (Chris
+ Nauroth via atm)
+
+ HDFS-4799. Corrupt replica can be prematurely removed from
+ corruptReplicas map. (todd via kihwal)
+
+ HDFS-4751. TestLeaseRenewer#testThreadName flakes. (Andrew Wang via atm)
+
+ HDFS-4533. start-dfs.sh ignores additional parameters besides -upgrade.
+ (Fengdong Yu via suresh)
+
+ HDFS-4765. Permission check of symlink deletion incorrectly throws
+ UnresolvedLinkException. (Andrew Wang via atm)
+
+ HDFS-4300. TransferFsImage.downloadEditsToStorage should use a tmp file for
+ destination. (Andrew Wang via atm)
+
+ HDFS-4813. Add volatile to BlocksMap.blocks so that the replication thread
+ can see the updated value. (Jing Zhao via szetszwo)
+
+ HDFS-3180. Add socket timeouts to WebHdfsFileSystem. (Chris Nauroth via
+ szetszwo)
+
+ HDFS-4787. Create a new HdfsConfiguration before each TestDFSClientRetries
+ testcases. (Tian Hong Wang via atm)
+
+ HDFS-4830. Typo in config settings for AvailableSpaceVolumeChoosingPolicy
+ in hdfs-default.xml. (atm)
+
+ HDFS-4824. FileInputStreamCache.close leaves dangling reference to
+ FileInputStreamCache.cacheCleaner. (Colin Patrick McCabe via todd)
+
+ HDFS-4298. StorageRetentionManager spews warnings when used with QJM. (atm)
+
+ HDFS-4725. Fix HDFS file handle leaks in FSEditLog, NameNode,
+ OfflineEditsBinaryLoader and some tests. (Chris Nauroth via szetszwo)
+
+ HDFS-4825. webhdfs / httpfs tests broken because of min block size change.
+ (Andrew Wang via suresh)
+
+ HDFS-4780. Use the correct relogin method for services. (Robert Parker via
+ kihwal)
+
+ HDFS-4827. Slight update to the implementation of API for handling favored
+ nodes in DFSClient (ddas)
+
+ HDFS-4865. Remove sub resource warning from httpfs log at startup time.
+ (ywskycn via tucu)
+
+ HDFS-4240. For nodegroup-aware block placement, when a node is excluded,
+ the nodes in the same nodegroup should also be excluded. (Junping Du
+ via szetszwo)
+
+ HDFS-4261. Fix bugs in Balaner causing infinite loop and
+ TestBalancerWithNodeGroup timeing out. (Junping Du via szetszwo)
+
+ HDFS-4382. Fix typo MAX_NOT_CHANGED_INTERATIONS. (Ted Yu via suresh)
+
+ HDFS-4840. ReplicationMonitor gets NPE during shutdown. (kihwal)
+
+ HADOOP-8957 HDFS tests for AbstractFileSystem#IsValidName should be overridden for
+ embedded file systems like ViewFs (Chris Nauroth via Sanjay Radia)
+
+ HDFS-4586. TestDataDirs.testGetDataDirsFromURIs fails with all directories
+ in dfs.datanode.data.dir are invalid. (Ivan Mitic via atm)
+
+ HDFS-3792. Fix two findbugs introduced by HDFS-3695 (todd)
+
+ HADOOP-9635 Fix potential Stack Overflow in DomainSocket.c (V. Karthik Kumar
+ via cmccabe)
+
+ HDFS-3163. TestHDFSCLI.testAll fails if the user name is not all lowercase.
+ (Brandon Li via atm)
+
+ HDFS-4845. FSNamesystem.deleteInternal should acquire write-lock before
+ changing the inode map. (Arpit Agarwal via szetszwo)
+
+ HDFS-4910. TestPermission failed in branch-2. (Chuan Liu via cnauroth)
+
+ HDFS-4906. HDFS Output streams should not accept writes after being
+ closed. (atm)
+
+ HDFS-4917. Start-dfs.sh cannot pass the parameters correctly.
+ (Fengdong Yu via suresh)
+
+ HDFS-4205. fsck fails with symlinks. (jlowe)
+
+ HDFS-4927. CreateEditsLog creates inodes with an invalid inode ID, which then
+ cannot be loaded by a namenode. (cnauroth)
+
+ HDFS-4944. WebHDFS cannot create a file path containing characters that must
+ be URI-encoded, such as space. (cnauroth)
+
+ HDFS-4888. Refactor and fix FSNamesystem.getTurnOffTip. (Ravi Prakash via
+ kihwal)
+
+ HDFS-4943. WebHdfsFileSystem does not work when original file path has
+ encoded chars. (Jerry He via szetszwo)
+
+ HDFS-4948. mvn site for hadoop-hdfs-nfs fails. (brandonli)
+
+ HDFS-4954. In nfs, OpenFileCtx.getFlushedOffset() should handle IOException.
+ (Brandon Li via szetszwo)
+
+ HDFS-4887. TestNNThroughputBenchmark exits abruptly. (kihwal)
+
+ HDFS-4980. Incorrect logging.properties file for hadoop-httpfs.
+ (Mark Grover via suresh)
+
+ HDFS-4999. Fix TestShortCircuitLocalRead on branch-2. (cmccabe via kihwal)
+
+ HDFS-4687. TestDelegationTokenForProxyUser#testWebHdfsDoAs is flaky with
+ JDK7. (Andrew Wang via atm)
+
+ HDFS-5003. TestNNThroughputBenchmark failed caused by existing directories.
+ (Xi Fang via cnauroth)
+
+ HDFS-5018. Misspelled DFSConfigKeys#DFS_NAMENODE_STALE_DATANODE_INTERVAL_DEFAULT
+ in javadoc of DatanodeInfo#isStale(). (Ted Yu via jing9)
+
+ HDFS-4602. TestBookKeeperHACheckpoints fails. (umamahesh)
+
+ HDFS-5016. Deadlock in pipeline recovery causes Datanode to be marked dead.
+ (suresh)
+
+ HDFS-5228. The RemoteIterator returned by DistributedFileSystem.listFiles
+ may throw NullPointerException. (szetszwo and cnauroth via szetszwo)
+
+ BREAKDOWN OF HDFS-347 SUBTASKS AND RELATED JIRAS
+
+ HDFS-4353. Encapsulate connections to peers in Peer and PeerServer classes.
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4354. Create DomainSocket and DomainPeer and associated unit tests.
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4356. BlockReaderLocal should use passed file descriptors rather than paths.
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4388. DomainSocket should throw AsynchronousCloseException when appropriate.
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4390. Bypass UNIX domain socket unit tests when they cannot be run.
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4400. DFSInputStream#getBlockReader: last retries should ignore the cache
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4401. Fix bug in DomainSocket path validation
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4402. Some small DomainSocket fixes: avoid findbugs warning, change
+ log level, etc. (Colin Patrick McCabe via todd)
+
+ HDFS-4418. increase default FileInputStreamCache size (todd)
+
+ HDFS-4416. Rename dfs.datanode.domain.socket.path to dfs.domain.socket.path
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4417. Fix case where local reads get disabled incorrectly
+ (Colin Patrick McCabe and todd via todd)
+
+ HDFS-4433. Make TestPeerCache not flaky (Colin Patrick McCabe via todd)
+
+ HDFS-4438. TestDomainSocket fails when system umask is set to 0002. (Colin
+ Patrick McCabe via atm)
+
+ HDFS-4440. Avoid annoying log message when dfs.domain.socket.path is not
+ set. (Colin Patrick McCabe via atm)
+
+ HDFS-4473. Don't create domain socket unless we need it. (Colin Patrick McCabe via atm)
+
+ HDFS-4485. DN should chmod socket path a+w. (Colin Patrick McCabe via atm)
+
+ HDFS-4453. Make a simple doc to describe the usage and design of the
+ shortcircuit read feature. (Colin Patrick McCabe via atm)
+
+ HDFS-4496. DFSClient: don't create a domain socket unless we need it (Colin
+ Patrick McCabe via todd)
+
+ HDFS-347: style cleanups (Colin Patrick McCabe via atm)
+
+ HDFS-4538. Allow use of legacy blockreader (Colin Patrick McCabe via todd)
+
+ HDFS-4661. A few little code cleanups of some HDFS-347-related code. (Colin
+ Patrick McCabe via atm)
+
+ BREAKDOWN OF HADOOP-8562 and HDFS-3602 SUBTASKS AND RELATED JIRAS
+
+ HDFS-4145. Merge hdfs cmd line scripts from branch-1-win. (David Lao,
+ Bikas Saha, Lauren Yang, Chuan Liu, Thejas M Nair and Ivan Mitic via suresh)
+
+ HDFS-4163. HDFS distribution build fails on Windows. (Chris Nauroth via
+ suresh)
+
+ HDFS-4316. branch-trunk-win contains test code accidentally added during
+ work on fixing tests on Windows. (Chris Nauroth via suresh)
+
+ HDFS-4297. Fix issues related to datanode concurrent reading and writing on
+ Windows. (Arpit Agarwal, Chuan Liu via suresh)
+
+ HDFS-4573. Fix TestINodeFile on Windows. (Arpit Agarwal via suresh)
+
+ HDFS-4572. Fix TestJournal failures on Windows. (Arpit Agarwal via suresh)
+
+ HDFS-4287. HTTPFS tests fail on Windows. (Chris Nauroth via suresh)
+
+ HDFS-4593. TestSaveNamespace fails on Windows. (Arpit Agarwal via suresh)
+
+ HDFS-4582. TestHostsFiles fails on Windows. (Ivan Mitic via suresh)
+
+ HDFS-4603. TestMiniDFSCluster fails on Windows. (Ivan Mitic via suresh)
+
+ HDFS-4604. TestJournalNode fails on Windows. (Ivan Mitic via suresh)
+
+ HDFS-4607. In TestGetConf.testGetSpecificKey(), use a platform-specific
+ line separator; otherwise, it fails on Windows. (Ivan Mitic via szetszwo)
+
+ HDFS-4625. Make TestNNWithQJM#testNewNamenodeTakesOverWriter work on
+ Windows. (Ivan Mitic via suresh)
+
+ HDFS-4674. TestBPOfferService fails on Windows due to failure parsing
+ datanode data directory as URI. (Chris Nauroth via suresh)
+
+ HDFS-4615. Fix TestDFSShell failures on Windows. (Arpit Agarwal
+ via szetszwo)
+
+ HDFS-4584. Skip TestNNWithQJM.testNewNamenodeTakesOverWriter() on Windows.
+ (Arpit Agarwal via szetszwo)
+
+ HDFS-4732. Fix TestDFSUpgradeFromImage which fails on Windows due to
+ failure to unpack old image tarball that contains hard links.
+ (Chris Nauroth via szetszwo)
+
+ HDFS-4741. TestStorageRestore#testStorageRestoreFailure fails on Windows.
+ (Arpit Agarwal via suresh)
+
+ HDFS-4743. TestNNStorageRetentionManager fails on Windows.
+ (Chris Nauroth via suresh)
+
+ HDFS-4740. Fixes for a few test failures on Windows.
+ (Arpit Agarwal via suresh)
+
+ HDFS-4722. TestGetConf#testFederation times out on Windows.
+ (Ivan Mitic via suresh)
+
+ HDFS-4705. Address HDFS test failures on Windows because of invalid
+ dfs.namenode.name.dir. (Ivan Mitic via suresh)
+
+ HDFS-4734. HDFS Tests that use ShellCommandFencer are broken on Windows.
+ (Arpit Agarwal via suresh)
+
+ HDFS-4610. Use common utils FileUtil#setReadable/Writable/Executable and
+ FileUtil#canRead/Write/Execute. (Ivan Mitic via suresh)
+
+ HDFS-4677. Editlog should support synchronous writes. (ivanmi)
+
+ HDFS-4752. TestRBWBlockInvalidation fails on Windows due to file locking.
+ (Chris Nauroth via suresh)
+
+ HDFS-4783. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails
+ on Windows. (cnauroth)
+
+ HDFS-4818. Several HDFS tests that attempt to make directories unusable do
+ not work correctly on Windows. (cnauroth)
+
+ BREAKDOWN OF HDFS-2802 HDFS SNAPSHOT SUBTASKS AND RELATED JIRAS
+
+ HDFS-4076. Support snapshot of single files. (szetszwo)
+
+ HDFS-4082. Add editlog opcodes for snapshot create and delete operations.
+ (suresh via szetszwo)
+
+ HDFS-4086. Add editlog opcodes to allow and disallow snapshots on a
+ directory. (Brandon Li via suresh)
+
+ HDFS-4083. Protocol changes for snapshots. (suresh)
+
+ HDFS-4077. Add support for Snapshottable Directory. (szetszwo via suresh)
+
+ HDFS-4087. Protocol changes for listSnapshots functionality.
+ (Brandon Li via suresh)
+
+ HDFS-4079. Add SnapshotManager which maintains a list for all the
+ snapshottable directories and supports snapshot methods such as setting a
+ directory to snapshottable and creating a snapshot. (szetszwo)
+
+ HDFS-4078. Handle replication in snapshots. (szetszwo)
+
+ HDFS-4084. Provide CLI support to allow and disallow snapshot
+ on a directory. (Brondon Li via suresh)
+
+ HDFS-4091. Add snapshot quota to limit the number of snapshots allowed.
+ (szetszwo)
+
+ HDFS-4097. Provide CLI support for createSnapshot. (Brandon Li via suresh)
+
+ HDFS-4092. Update file deletion logic for snapshot so that the current inode
+ is removed from the circular linked list; and if some blocks at the end of
+ the block list no longer belong to any other inode, collect them and update
+ the block list. (szetszwo)
+
+ HDFS-4111. Support snapshot of subtrees. (szetszwo via suresh)
+
+ HDFS-4119. Complete the allowSnapshot code and add a test for it. (szetszwo)
+
+ HDFS-4133. Add testcases for testing basic snapshot functionalities.
+ (Jing Zhao via suresh)
+
+ HDFS-4116. Add auditlog for some snapshot operations. (Jing Zhao via suresh)
+
+ HDFS-4095. Add some snapshot related metrics. (Jing Zhao via suresh)
+
+ HDFS-4141. Support directory diff - the difference between the current state
+ and a previous snapshot of an INodeDirectory. (szetszwo)
+
+ HDFS-4146. Use getter and setter in INodeFileWithLink to access blocks and
+ initialize root directory as snapshottable. (szetszwo)
+
+ HDFS-4149. Implement the disallowSnapshot(..) in FSNamesystem and add
+ resetSnapshottable(..) to SnapshotManager. (szetszwo)
+
+ HDFS-4147. When there is a snapshot in a subtree, deletion of the subtree
+ should fail. (Jing Zhao via szetszwo)
+
+ HDFS-4150. Update the inode in the block map when a snapshotted file or a
+ snapshot file is deleted. (Jing Zhao via szetszwo)
+
+ HDFS-4159. Rename should fail when the destination directory is
+ snapshottable and has snapshots. (Jing Zhao via szetszwo)
+
+ HDFS-4170. Add snapshot information to INodesInPath. (szetszwo)
+
+ HDFS-4177. Add a snapshot parameter to INodeDirectory.getChildrenList() for
+ selecting particular snapshot children list views. (szetszwo)
+
+ HDFS-4148. Disallow write/modify operations on files and directories in a
+ snapshot. (Brandon Li via suresh)
+
+ HDFS-4188. Add Snapshot.ID_COMPARATOR for comparing IDs and fix a bug in
+ ReadOnlyList.Util.binarySearch(..). (szetszwo)
+
+ HDFS-4187. Add tests for replication handling in snapshots. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4196. Support renaming of snapshots. (Jing Zhao via szetszwo)
+
+ HDFS-4175. Additional snapshot tests for more complicated directory
+ structure and modifications. (Jing Zhao via suresh)
+
+ HDFS-4293. Fix TestSnapshot failure. (Jing Zhao via suresh)
+
+ HDFS-4317. Change INode and its subclasses to support HDFS-4103. (szetszwo)
+
+ HDFS-4103. Support O(1) snapshot creation. (szetszwo)
+
+ HDFS-4330. Support snapshots up to the snapshot limit. (szetszwo)
+
+ HDFS-4357. Fix a bug that if an inode is replaced, further INode operations
+ should apply to the new inode. (Jing Zhao via szetszwo)
+
+ HDFS-4230. Support listing of all the snapshottable directories. (Jing Zhao
+ via szetszwo)
+
+ HDFS-4244. Support snapshot deletion. (Jing Zhao via szetszwo)
+
+ HDFS-4245. Include snapshot related operations in TestOfflineEditsViewer.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4395. In INodeDirectorySnapshottable's constructor, the passed-in dir
+ could be an INodeDirectoryWithSnapshot. (Jing Zhao via szetszwo)
+
+ HDFS-4397. Fix a bug in INodeDirectoryWithSnapshot.Diff.combinePostDiff(..)
+ that it may put the wrong node into the deleted list. (szetszwo)
+
+ HDFS-4407. Change INodeDirectoryWithSnapshot.Diff.combinePostDiff(..) to
+ merge-sort like and keep the postDiff parameter unmodified. (szetszwo)
+
+ HDFS-4098. Add FileWithSnapshot, INodeFileUnderConstructionWithSnapshot and
+ INodeFileUnderConstructionSnapshot for supporting append to snapshotted
+ files. (szetszwo)
+
+ HDFS-4126. Add reading/writing snapshot information to FSImage.
+ (Jing Zhao via suresh)
+
+ HDFS-4436. Change INode.recordModification(..) to return only the current
+ inode and remove the updateCircularList parameter from some methods in
+ INodeDirectoryWithSnapshot.Diff. (szetszwo)
+
+ HDFS-4429. When the latest snapshot exists, INodeFileUnderConstruction
+ should be replaced with INodeFileWithSnapshot but not INodeFile.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4441. Move INodeDirectoryWithSnapshot.Diff and the related classes to a
+ package. (szetszwo)
+
+ HDFS-4432. Support INodeFileUnderConstructionWithSnapshot in FSImage
+ saving/loading. (Jing Zhao via suresh)
+
+ HDFS-4131. Add capability to namenode to get snapshot diff. (Jing Zhao via
+ suresh)
+
+ HDFS-4447. Refactor INodeDirectoryWithSnapshot for supporting general INode
+ diff lists. (szetszwo)
+
+ HDFS-4189. Renames the getMutableXxx methods to getXxx4Write and fix a bug
+ that some getExistingPathINodes calls should be getINodesInPath4Write.
+ (szetszwo)
+
+ HDFS-4361. When listing snapshottable directories, only return those
+ where the user has permission to take snapshots. (Jing Zhao via szetszwo)
+
+ HDFS-4464. Combine collectSubtreeBlocksAndClear with deleteDiffsForSnapshot
+ and rename it to destroySubtreeAndCollectBlocks. (szetszwo)
+
+ HDFS-4414. Add support for getting snapshot diff from DistributedFileSystem.
+ (Jing Zhao via suresh)
+
+ HDFS-4446. Support file snapshots with diff lists. (szetszwo)
+
+ HDFS-4480. Eliminate the file snapshot circular linked list. (szetszwo)
+
+ HDFS-4481. Change fsimage to support snapshot file diffs. (szetszwo)
+
+ HDFS-4500. Refactor snapshot INode methods. (szetszwo)
+
+ HDFS-4487. Fix snapshot diff report for HDFS-4446. (Jing Zhao via szetszwo)
+
+ HDFS-4431. Support snapshot in OfflineImageViewer. (Jing Zhao via szetszwo)
+
+ HDFS-4503. Update computeContentSummary(..), spaceConsumedInTree(..) and
+ diskspaceConsumed(..) in INode for snapshot. (szetszwo)
+
+ HDFS-4499. Fix file/directory/snapshot deletion for file diff. (Jing Zhao
+ via szetszwo)
+
+ HDFS-4524. Update SnapshotManager#snapshottables when loading fsimage.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4520. Support listing snapshots under a snapshottable directory using
+ ls. (Jing Zhao via szetszwo)
+
+ HDFS-4514. Add CLI for supporting snapshot rename, diff report, and
+ snapshottable directory listing. (Jing Zhao via szetszwo)
+
+ HDFS-4523. Fix INodeFile replacement, TestQuota and javac errors from trunk
+ merge. (szetszwo)
+
+ HDFS-4507. Update quota verification for snapshots. (szetszwo)
+
+ HDFS-4545. With snapshots, FSDirectory.unprotectedSetReplication(..) always
+ changes file replication but it may or may not changes block replication.
+ (szetszwo)
+
+ HDFS-4557. Fix FSDirectory#delete when INode#cleanSubtree returns 0.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4579. Annotate snapshot tests. (Arpit Agarwal via suresh)
+
+ HDFS-4574. Move Diff to the util package. (szetszwo)
+
+ HDFS-4563. Update namespace/diskspace usage after deleting snapshots.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4144. Create test for all snapshot-related metrics.
+ (Jing Zhao via suresh)
+
+ HDFS-4556. Add snapshotdiff and LsSnapshottableDir tools to hdfs script.
+ (Arpit Agarwal via szetszwo)
+
+ HDFS-4534. Add INodeReference in order to support rename with snapshots.
+ (szetszwo)
+
+ HDFS-4616. Update the FilesDeleted metric while deleting file/dir in the
+ current tree. (Jing Zhao via szetszwo)
+
+ HDFS-4627. Fix FSImageFormat#Loader NPE and synchronization issues.
+ (Jing Zhao via suresh)
+
+ HDFS-4612. Not to use INode.getParent() when generating snapshot diff
+ report. (Jing Zhao via szetszwo)
+
+ HDFS-4636. Update quota usage when deleting files/dirs that were created
+ after taking the latest snapshot. (Jing Zhao via szetszwo)
+
+ HDFS-4648. For snapshot deletion, when merging the diff from to-delete
+ snapshot to the prior snapshot, make sure files/directories created after
+ the prior snapshot get deleted. (Jing Zhao via szetszwo)
+
+ HDFS-4637. INodeDirectory#replaceSelf4Quota may incorrectly convert a newly
+ created directory to an INodeDirectoryWithSnapshot. (Jing Zhao via szetszwo)
+
+ HDFS-4611. Update FSImage for INodeReference. (szetszwo)
+
+ HDFS-4647. Rename should call setLocalName after an inode is removed from
+ snapshots. (Arpit Agarwal via szetszwo)
+
+ HDFS-4684. Use INode id for image serialization when writing INodeReference.
+ (szetszwo)
+
+ HDFS-4675. Fix rename across snapshottable directories. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4692. Use timestamp as default snapshot names. (szetszwo)
+
+ HDFS-4666. Define ".snapshot" as a reserved inode name so that users cannot
+ create a file/directory with ".snapshot" as the name. If ".snapshot" is
+ used in a previous version of HDFS, it must be renamed before upgrade;
+ otherwise, upgrade will fail. (szetszwo)
+
+ HDFS-4700. Fix the undo section of rename with snapshots. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4529. Disallow concat when one of the src files is in some snapshot.
+ (szetszwo)
+
+ HDFS-4550. Refactor INodeDirectory.INodesInPath to a standalone class.
+ (szetszwo)
+
+ HDFS-4707. Add snapshot methods to FilterFileSystem and fix findbugs
+ warnings. (szetszwo)
+
+ HDFS-4706. Do not replace root inode for disallowSnapshot. (szetszwo)
+
+ HDFS-4717. Change the path parameter type of the snapshot methods in
+ HdfsAdmin from String to Path. (szetszwo)
+
+ HDFS-4708. Add snapshot user documentation. (szetszwo)
+
+ HDFS-4726. Fix test failures after merging the INodeId-INode mapping
+ from trunk. (Jing Zhao via szetszwo)
+
+ HDFS-4727. Update inodeMap after deleting files/directories/snapshots.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4719. Remove AbstractINodeDiff.Factory and move its methods to
+ AbstractINodeDiffList. (Arpit Agarwal via szetszwo)
+
+ HDFS-4735. DisallowSnapshot throws IllegalStateException for nested
+ snapshottable directories. (Jing Zhao via szetszwo)
+
+ HDFS-4738. Changes AbstractINodeDiff to implement Comparable<Integer>, and
+ fix javadoc and other warnings. (szetszwo)
+
+ HDFS-4686. Update quota computation for rename and INodeReference.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4729. Fix OfflineImageViewer and permission checking for snapshot
+ operations. (Jing Zhao via szetszwo)
+
+ HDFS-4749. Use INodeId to identify the corresponding directory node in
+ FSImage saving/loading. (Jing Zhao via szetszwo)
+
+ HDFS-4742. Fix appending to a renamed file with snapshot. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4755. Fix AccessControlException message and moves "implements
+ LinkedElement" from INode to INodeWithAdditionalFields. (szetszwo)
+
+ HDFS-4650. Fix a bug in FSDirectory and add more unit tests for rename with
+ existence of snapshottable directories and snapshots. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4650. When passing two non-existing snapshot names to snapshotDiff, it
+ returns success if the names are the same. (Jing Zhao via szetszwo)
+
+ HDFS-4767. If a directory is snapshottable, do not replace it when clearing
+ quota. (Jing Zhao via szetszwo)
+
+ HDFS-4578. Restrict snapshot IDs to 24-bit wide. (Arpit Agarwal via
+ szetszwo)
+
+ HDFS-4773. Fix bugs in quota usage computation and OfflineImageViewer.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4760. Update inodeMap after node replacement. (Jing Zhao via szetszwo)
+
+ HDFS-4758. Disallow nested snapshottable directories and unwrap
+ RemoteException. (szetszwo)
+
+ HDFS-4781. Fix a NullPointerException when listing .snapshot under
+ a non-existing directory. (szetszwo)
+
+ HDFS-4791. Update and fix deletion of reference inode. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4798. Update computeContentSummary() for the reference nodes in
+ snapshots. (szetszwo)
+
+ HDFS-4800. Fix INodeDirectoryWithSnapshot#cleanDeletedINode. (Jing Zhao via
+ szetszwo)
+
+ HDFS-4801. lsSnapshottableDir throws IllegalArgumentException when root is
+ snapshottable. (Jing Zhao via szetszwo)
+
+ HDFS-4802. Disallowing snapshot on / twice should throw SnapshotException
+ but not IllegalStateException. (Jing Zhao via szetszwo)
+
+ HDFS-4806. In INodeDirectoryWithSnapshot, use isInLatestSnapshot() to
+ determine if an added/removed child should be recorded in the snapshot diff.
+ (Jing Zhao via szetszwo)
+
+ HDFS-4809. When a QuotaExceededException is thrown during rename, the quota
+ usage should be subtracted back. (Jing Zhao via szetszwo)
+
+ HDFS-4842. Identify the correct prior snapshot when deleting a
+ snapshot under a renamed subtree. (jing9)
+
+ HDFS-4846. Clean up snapshot CLI commands output stacktrace for invalid
+ arguments. (Jing Zhao via brandonli)
+
+ HDFS-4857. Snapshot.Root and AbstractINodeDiff#snapshotINode should not be
+ put into INodeMap when loading FSImage. (jing9)
+
+ HDFS-4863. The root directory should be added to the snapshottable
+ directory list while loading fsimage. (jing9)
+
+ HDFS-4848. copyFromLocal and renaming a file to ".snapshot" should output
+ that ".snapshot" is a reserved name. (Jing Zhao via brandonli)
+
+ HDFS-4826. TestNestedSnapshots times out due to repeated slow edit log
+ flushes when running on virtualized disk. (Chris Nauroth via szetszwo)
+
+ HDFS-4876. Fix the javadoc of FileWithSnapshot and move FileDiffList to
+ FileWithSnapshot. (szetszwo)
+
+ HDFS-4850. Fix OfflineImageViewer to work on fsimages with empty files or
+ snapshots. (jing9)
+
+ HDFS-4877. Snapshot: fix the scenario where a directory is renamed under
+ its prior descendant. (jing9)
+
+ HDFS-4873. callGetBlockLocations returns incorrect number of blocks for
+ snapshotted files. (jing9)
+
+ HDFS-4819. Update Snapshot doc to clarify that nested snapshots are not
+ allowed. (szetszwo)
+
+ HDFS-4902. DFSClient.getSnapshotDiffReport should use string path rather
+ than o.a.h.fs.Path. (Binglin Chang via jing9)
+
+ HDFS-4875. Add a test for testing snapshot file length.
+ (Arpit Agarwal via jing9)
+
+ HDFS-4841. FsShell commands using secure webhfds fail ClientFinalizer
+ shutdown hook. (rkanter via tucu)
+
+ HDFS-4951. FsShell commands using secure httpfs throw exceptions due
+ to missing TokenRenewer. (rknater via tucu)
+
+ HDFS-4969. WebhdfsFileSystem expects non-standard WEBHDFS Json element.
+ (rkanter via tucu)
+
+ HDFS-4797. BlockScanInfo does not override equals(..) and hashCode()
+ consistently. (szetszwo)
+
+ HDFS-4978. Make disallowSnapshot idempotent. (jing9)
+
+ HDFS-5005. Move SnapshotException and SnapshotAccessControlException
+ to o.a.h.hdfs.protocol. (jing9)
+
+ HDFS-4982. JournalNode should relogin from keytab before fetching logs
+ from other JNs (todd)
+
+Release 2.0.5-alpha - 06/06/2013
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+Release 2.0.4-alpha - 2013-04-25
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+Release 2.0.3-alpha - 2013-02-06
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-4122. Cleanup HDFS logs and reduce the size of logged messages.
+ (suresh)
+
+ HDFS-4362. GetDelegationTokenResponseProto does not handle null token.
+ (suresh)
+
+ HDFS-4367. GetDataEncryptionKeyResponseProto does not handle null
+ response. (suresh)
+
+ HDFS-4364. GetLinkTargetResponseProto does not handle null path. (suresh)
+
+ HDFS-4369. GetBlockKeysResponseProto does not handle null response.
+ (suresh)
+
+ HDFS-4451. hdfs balancer command returns exit code 1 on success instead
+ of 0. (Joshua Blatt via suresh)
+
+ HDFS-4350. Make enabling of stale marking on read and write paths
+ independent. (Andrew Wang via suresh)
+
+
+ NEW FEATURES
+
+ HDFS-2656. Add libwebhdfs, a pure C client based on WebHDFS.
+ (Jaimin D Jetly and Jing Zhao via szetszwo)
+
+ HDFS-3912. Detect and avoid stale datanodes for writes.
+ (Jing Zhao via suresh)
+
+ HDFS-4059. Add number of stale DataNodes to metrics. (Jing Zhao via suresh)
+
+ HDFS-4155. libhdfs implementation of hsync API (Liang Xie via todd)
+
+ HDFS-4213. Add an API to hsync for updating the last block length at the
+ namenode. (Jing Zhao via szetszwo)
+
+ HDFS-3077. Implement QuorumJournalManager, a distributed mechanism for
+ reliably storing HDFS edit logs. See dedicated section below for breakdown
+ of subtasks.
+
+ IMPROVEMENTS
+
+ HDFS-3925. Prettify PipelineAck#toString() for printing to a log
+ (Andrew Wang via todd)
+
+ HDFS-3939. NN RPC address cleanup. (eli)
+
+ HDFS-3373. Change DFSClient input stream socket cache to global static and
+ add a thread to cleanup expired cache entries. (John George via szetszwo)
+
+ HDFS-3896. Add descriptions for dfs.namenode.rpc-address and
+ dfs.namenode.servicerpc-address to hdfs-default.xml. (Jeff Lord via atm)
+
+ HDFS-3996. Add debug log removed in HDFS-3873 back. (eli)
+
+ HDFS-3916. libwebhdfs (C client) code cleanups.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3813. Log error message if security and WebHDFS are enabled but
+ principal/keytab are not configured. (Stephen Chu via atm)
+
+ HDFS-3483. Better error message when hdfs fsck is run against a ViewFS
+ config. (Stephen Fritz via atm)
+
+ HDFS-3682. MiniDFSCluster#init should provide more info when it fails.
+ (todd via eli)
+
+ HDFS-4008. TestBalancerWithEncryptedTransfer needs a timeout. (eli)
+
+ HDFS-4007. Rehabilitate bit-rotted unit tests under
+ hadoop-hdfs-project/hadoop-hdfs/src/test/unit/
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4041. Hadoop HDFS Maven protoc calls must not depend on external
+ sh script. (Chris Nauroth via suresh)
+
+ HADOOP-8911. CRLF characters in source and text files.
+ (Raja Aluri via suresh)
+
+ HDFS-4037. Rename the getReplication() method in BlockCollection to
+ getBlockReplication(). (szetszwo)
+
+ HDFS-4036. Remove "throws UnresolvedLinkException" from
+ FSDirectory.unprotectedAddFile(..). (Jing Zhao via szetszwo)
+
+ HDFS-2946. HA: Put a cap on the number of completed edits files retained
+ by the NN. (atm)
+
+ HDFS-4029. GenerationStamp should use an AtomicLong. (eli)
+
+ HDFS-4068. DatanodeID and DatanodeInfo member should be private. (eli)
+
+ HDFS-4073. Two minor improvements to FSDirectory. (Jing Zhao via szetszwo)
+
+ HDFS-4074. Remove the unused default constructor from INode. (Brandon Li
+ via szetszwo)
+
+ HDFS-4088. Remove "throws QuotaExceededException" from an
+ INodeDirectoryWithQuota constructor. (szetszwo)
+
+ HDFS-4099. Clean up replication code and add more javadoc. (szetszwo)
+
+ HDFS-4107. Add utility methods for casting INode to INodeFile and
+ INodeFileUnderConstruction. (szetszwo)
+
+ HDFS-4112. A few improvements on INodeDirectory include adding a utility
+ method for casting; avoiding creation of new empty lists; cleaning up
+ some code and rewriting some javadoc. (szetszwo)
+
+ HDFS-4121. Add namespace declarations in hdfs .proto files for languages
+ other than java. (Binglin Chang via suresh)
+
+ HDFS-3573. Supply NamespaceInfo when instantiating JournalManagers.
+ (todd and ivank via umamahesh)
+
+ HDFS-3695. Genericize format() to non-file JournalManagers.
+ (todd via umamahesh)
+
+ HDFS-3789. JournalManager#format() should be able to throw IOException.
+ (Ivan Kelly via umamahesh)
+
+ HDFS-3809. Make BKJM use protobufs for all serialization with ZK.
+ (Ivan Kelly via umamhesh)
+
+ HDFS-3916. libwebhdfs testing code cleanup. (Jing Zhao via suresh)
+
+ HDFS-4143. Change blocks to private in INodeFile and renames isLink() to
+ isSymlink() in INode. (szetszwo)
+
+ HDFS-4046. Rename ChecksumTypeProto enum NULL since it is illegal in
+ C/C++. (Binglin Chang via suresh)
+
+ HDFS-4048. Use ERROR instead of INFO for volume failure logs.
+ (Stephen Chu via eli)
+
+ HDFS-1322. Document umask in DistributedFileSystem#mkdirs javadocs.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-4038. Override toString() for BookKeeperEditLogInputStream.
+ (Vinay via umamahesh)
+
+ HDFS-4214. OfflineEditsViewer should print out the offset at which it
+ encountered an error. (Colin Patrick McCabe via atm)
+
+ HDFS-4199. Provide test for HdfsVolumeId. (Ivan A. Veselovsky via atm)
+
+ HDFS-3049. During the normal NN startup process, fall back on a different
+ edit log if we see one that is corrupt (Colin Patrick McCabe via todd)
+
+ HDFS-3571. Allow EditLogFileInputStream to read from a remote URL (todd)
+
+ HDFS-4110. Refine a log printed in JNStorage. (Liang Xie via suresh)
+
+ HDFS-4153. Add START_MSG/SHUTDOWN_MSG for JournalNode. (liang xie via atm)
+
+ HDFS-3935. Add JournalNode to the start/stop scripts (Andy Isaacson via todd)
+
+ HDFS-4268. Remove redundant enum NNHAStatusHeartbeat.State. (shv)
+
+ HDFS-3680. Allow customized audit logging in HDFS FSNamesystem. (Marcelo
+ Vanzin via atm)
+
+ HDFS-4130. BKJM: The reading for editlog at NN starting using bkjm is not efficient.
+ (Han Xiao via umamahesh)
+
+ HDFS-4326. bump up Tomcat version for HttpFS to 6.0.36. (tucu via acmurthy)
+
+ HDFS-4270. Introduce soft and hard limits for max replication so that
+ replications of the highest priority are allowed to choose a source datanode
+ that has reached its soft limit but not the hard limit. (Derek Dagit via
+ szetszwo)
+
+ HADOOP-9173. Add security token protobuf definition to common and
+ use it in hdfs. (suresh)
+
+ HDFS-4030. BlockManager excessBlocksCount and
+ postponedMisreplicatedBlocksCount should be AtomicLongs. (eli)
+
+ HDFS-4031. Update findbugsExcludeFile.xml to include findbugs 2
+ exclusions. (eli)
+
+ HDFS-4033. Miscellaneous findbugs 2 fixes. (eli)
+
+ HDFS-4034. Remove redundant null checks. (eli)
+
+ HDFS-4035. LightWeightGSet and LightWeightHashSet increment a
+ volatile without synchronization. (eli)
+
+ HDFS-4032. Specify the charset explicitly rather than rely on the
+ default. (eli)
+
+ HDFS-4363. Combine PBHelper and HdfsProtoUtil and remove redundant
+ methods. (suresh)
+
+ HDFS-4377. Some trivial DN comment cleanup. (eli)
+
+ HDFS-4381. Document fsimage format details in FSImageFormat class javadoc.
+ (Jing Zhao via suresh)
+
+ HDFS-4375. Use token request messages defined in hadoop common.
+ (suresh)
+
+ HDFS-4392. Use NetUtils#getFreeSocketPort in MiniDFSCluster.
+ (Andrew Purtell via suresh)
+
+ HDFS-4393. Make empty request and responses in protocol translators can be
+ static final members. (Brandon Li via suresh)
+
+ HDFS-4403. DFSClient can infer checksum type when not provided by reading
+ first byte (todd)
+
+ HDFS-4259. Improve pipeline DN replacement failure message (harsh)
+
+ HDFS-3598. WebHDFS support for file concat. (Plamen Jeliazkov via shv)
+
+ HDFS-4456. Add concat to HttpFS and WebHDFS REST API docs. (plamenj2003 via tucu)
+
+ HDFS-3131. Improve TestStorageRestore. (Brandon Li via atm)
+
+ OPTIMIZATIONS
+
+ HDFS-3429. DataNode reads checksums even if client does not need them (todd)
+
+ BUG FIXES
+
+ HDFS-3919. MiniDFSCluster:waitClusterUp can hang forever.
+ (Andy Isaacson via eli)
+
+ HDFS-3924. Multi-byte id in HdfsVolumeId. (Andrew Wang via atm)
+
+ HDFS-3936. MiniDFSCluster shutdown races with BlocksMap usage. (eli)
+
+ HDFS-3951. datanode web ui does not work over HTTPS when datanode is started in secure mode. (tucu)
+
+ HDFS-3949. NameNodeRpcServer#join should join on both client and
+ server RPC servers. (eli)
+
+ HDFS-3932. NameNode Web UI broken if the rpc-address is set to the wildcard.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3931. TestDatanodeBlockScanner#testBlockCorruptionPolicy2 is broken.
+ (Andy Isaacson via eli)
+
+ HDFS-3964. Make NN log of fs.defaultFS debug rather than info. (eli)
+
+ HDFS-3992. Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown()
+ sometimes throws NPEs. (Ivan A. Veselovsky via atm)
+
+ HDFS-3753. Tests don't run with native libraries.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-4000. TestParallelLocalRead fails with "input ByteBuffers
+ must be direct buffers". (Colin Patrick McCabe via eli)
+
+ HDFS-3999. HttpFS OPEN operation expects len parameter, it should be length. (tucu)
+
+ HDFS-4006. TestCheckpoint#testSecondaryHasVeryOutOfDateImage
+ occasionally fails due to unexpected exit. (todd via eli)
+
+ HDFS-4018. testMiniDFSClusterWithMultipleNN is missing some
+ cluster cleanup. (eli)
+
+ HDFS-4020. TestRBWBlockInvalidation may time out. (eli)
+
+ HDFS-4021. Misleading error message when resources are low on the NameNode.
+ (Christopher Conner via atm)
+
+ HDFS-4044. Duplicate ChecksumType definition in HDFS .proto files.
+ (Binglin Chang via suresh)
+
+ HDFS-4049. Fix hflush performance regression due to nagling delays
+ (todd)
+
+ HDFS-3678. Edit log files are never being purged from 2NN. (atm)
+
+ HDFS-4058. DirectoryScanner may fail with IOOB if the directory
+ scanning threads return out of volume order. (eli)
+
+ HDFS-3985. Add timeouts to TestMulitipleNNDataBlockScanner. (todd via eli)
+
+ HDFS-4061. TestBalancer and TestUnderReplicatedBlocks need timeouts. (eli)
+
+ HDFS-3997. OfflineImageViewer incorrectly passes value of imageVersion when
+ visiting IS_COMPRESSED element. (Mithun Radhakrishnan via atm)
+
+ HDFS-4055. TestAuditLogs is flaky. (Binglin Chang via eli)
+
+ HDFS-4072. On file deletion remove corresponding blocks pending
+ replications. (Jing Zhao via suresh)
+
+ HDFS-4022. Replication not happening for appended block.
+ (Vinay via umamahesh)
+
+ HDFS-3948. Do not use hflush in TestWebHDFS.testNamenodeRestart() since the
+ out stream returned by WebHdfsFileSystem does not support it. (Jing Zhao
+ via szetszwo)
+
+ HDFS-3616. Fix a ConcurrentModificationException bug that BP actor threads
+ may not be shutdown properly in DataNode. (Jing Zhao via szetszwo)
+
+ HDFS-4127. Log message is not correct in case of short of replica.
+ (Junping Du via suresh)
+
+ HADOOP-8994. TestDFSShell creates file named "noFileHere", making further
+ tests hard to understand (Andy Isaacson via daryn)
+
+ HDFS-3804. TestHftpFileSystem fails intermittently with JDK7
+ (Trevor Robinson via daryn)
+
+ HDFS-4132. When libwebhdfs is not enabled, nativeMiniDfsClient frees
+ uninitialized memory (Colin Patrick McCabe via todd)
+
+ HDFS-1331. dfs -test should work like /bin/test (Andy Isaacson via daryn)
+
+ HDFS-3979. For hsync, datanode should wait for the local sync to complete
+ before sending ack. (Lars Hofhansl via szetszwo)
+
+ HDFS-3810. Implement format() for BKJM (Ivan Kelly via umamahesh)
+
+ HDFS-3625. Fix TestBackupNode by properly initializing edit log during
+ startup. (Junping Du via todd)
+
+ HDFS-4138. BackupNode startup fails due to uninitialized edit log.
+ (Kihwal Lee via shv)
+
+ HDFS-4162. Some malformed and unquoted HTML strings are returned from
+ datanode web ui. (Darek Dagit via suresh)
+
+ HDFS-4164. fuse_dfs: add -lrt to the compiler command line on Linux.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3921. NN will prematurely consider blocks missing when entering active
+ state while still in safe mode. (atm)
+
+ HDFS-4106. BPServiceActor#lastHeartbeat, lastBlockReport and
+ lastDeletedReport should be volatile. (Jing Zhao via suresh)
+
+ HDFS-4139. fuse-dfs RO mode still allows file truncation.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-4104. dfs -test -d prints inappropriate error on nonexistent directory
+ (Andy Isaacson via daryn)
+
+ HDFS-3623. BKJM: zkLatchWaitTimeout hard coded to 6000. Make use of ZKSessionTimeout instead.
+ (umamahesh)
+
+ HDFS-4100. Fix all findbug security warings. (Liang Xie via eli)
+
+ HDFS-3507. DFS#isInSafeMode needs to execute only on Active NameNode.
+ (Vinay via atm)
+
+ HDFS-4105. The SPNEGO user for secondary namenode should use the web
+ keytab. (Arpit Gupta via jitendra)
+
+ HDFS-4156. Seeking to a negative position should throw an IOE.
+ (Eli Reisman via eli)
+
+ HDFS-4171. WebHDFS and HttpFs should accept only valid Unix user
+ names. (tucu)
+
+ HDFS-4178. Shell scripts should not close stderr (Andy Isaacson via daryn)
+
+ HDFS-4179. BackupNode: allow reads, fix checkpointing, safeMode. (shv)
+
+ HDFS-4216. Do not ignore QuotaExceededException when adding symlinks.
+ (szetszwo)
+
+ HDFS-4242. Map.Entry is incorrectly used in LeaseManager since the behavior
+ of it is undefined after the iteration or modifications of the map.
+ (szetszwo)
+
+ HDFS-4231. BackupNode: Introduce BackupState. (shv)
+
+ HDFS-4238. Standby namenode should not do purging of shared
+ storage edits. (todd)
+
+ HDFS-4282. TestEditLog.testFuzzSequences FAILED in all pre-commit test
+ (todd)
+
+ HDFS-4236. Remove artificial limit on username length introduced in
+ HDFS-4171. (tucu via suresh)
+
+ HDFS-4279. NameNode does not initialize generic conf keys when started
+ with -recover. (Colin Patrick McCabe via atm)
+
+ HDFS-4291. edit log unit tests leave stray test_edit_log_file around
+ (Colin Patrick McCabe via todd)
+
+ HDFS-4292. Sanity check not correct in RemoteBlockReader2.newBlockReader
+ (Binglin Chang via todd)
+
+ HDFS-4295. Using port 1023 should be valid when starting Secure DataNode
+ (Stephen Chu via todd)
+
+ HDFS-4294. Backwards compatibility is not maintained for TestVolumeId.
+ (Ivan A. Veselovsky and Robert Parker via atm)
+
+ HDFS-2264. NamenodeProtocol has the wrong value for clientPrincipal in
+ KerberosInfo annotation. (atm)
+
+ HDFS-4307. SocketCache should use monotonic time. (Colin Patrick McCabe
+ via atm)
+
+ HDFS-4315. DNs with multiple BPs can have BPOfferServices fail to start
+ due to unsynchronized map access. (atm)
+
+ HDFS-4140. fuse-dfs handles open(O_TRUNC) poorly. (Colin Patrick McCabe
+ via atm)
+
+ HDFS-4308. addBlock() should persist file blocks once.
+ (Plamen Jeliazkov via shv)
+
+ HDFS-4347. Avoid infinite waiting checkpoint to complete in TestBackupNode.
+ (Plamen Jeliazkov via shv)
+
+ HDFS-4349. Add test for reading files from BackupNode. (shv)
+
+ HDFS-4302. Fix fatal exception when starting NameNode with DEBUG logs
+ (Eugene Koontz via todd)
+
+ HDFS-3970. Fix bug causing rollback of HDFS upgrade to result in bad
+ VERSION file. (Vinay and Andrew Wang via atm)
+
+ HDFS-4306. PBHelper.convertLocatedBlock miss convert BlockToken. (Binglin
+ Chang via atm)
+
+ HDFS-4384. test_libhdfs_threaded gets SEGV if JNIEnv cannot be
+ initialized. (Colin Patrick McCabe via eli)
+
+ HDFS-4328. TestLargeBlock#testLargeBlockSize is timing out. (Chris Nauroth
+ via atm)
+
+ HDFS-4274. BlockPoolSliceScanner does not close verification log during
+ shutdown. (Chris Nauroth via suresh)
+
+ HDFS-1245. Pluggable block id generation. (shv)
+
+ HDFS-4415. HostnameFilter should handle hostname resolution failures and
+ continue processing. (Robert Kanter via atm)
+
+ HDFS-4359. Slow RPC responses from NN can prevent metrics collection on
+ DNs. (liang xie via atm)
+
+ HDFS-4444. Add space between total transaction time and number of
+ transactions in FSEditLog#printStatistics. (Stephen Chu via suresh)
+
+ HDFS-4428. FsDatasetImpl should disclose what the error is when a rename
+ fails. (Colin Patrick McCabe via atm)
+
+ HDFS-4452. getAdditionalBlock() can create multiple blocks if the client
+ times out and retries. (shv)
+
+ HDFS-4445. All BKJM ledgers are not checked while tailing, So failover will fail.
+ (Vinay via umamahesh)
+
+ HDFS-4462. 2NN will fail to checkpoint after an HDFS upgrade from a
+ pre-federation version of HDFS. (atm)
+
+ HDFS-4404. Create file failure when the machine of first attempted NameNode
+ is down. (Todd Lipcon via atm)
+
+ HDFS-4344. dfshealth.jsp throws NumberFormatException when
+ dfs.hosts/dfs.hosts.exclude includes port number. (Andy Isaacson via atm)
+
+ HDFS-4468. Use the new StringUtils methods added by HADOOP-9252 and fix
+ TestHDFSCLI and TestQuota. (szetszwo)
+
+ HDFS-4458. In DFSUtil.getNameServiceUris(..), convert default fs URI using
+ NetUtils.createSocketAddr(..) for being consistent with other addresses.
+ (Binglin Chang via szetszwo)
+
+ BREAKDOWN OF HDFS-3077 SUBTASKS
+
+ HDFS-3077. Quorum-based protocol for reading and writing edit logs.
+ (todd, Brandon Li, and Hari Mankude via todd)
+
+ HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary (todd)
+
+ HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs
+ (todd)
+
+ HDFS-3693. JNStorage should read its storage info even before a writer
+ becomes active (todd)
+
+ HDFS-3725. Fix QJM startup when individual JNs have gaps (todd)
+
+ HDFS-3741. Exhaustive failure injection test for skipped RPCs (todd)
+
+ HDFS-3773. TestNNWithQJM fails after HDFS-3741. (atm)
+
+ HDFS-3793. Implement genericized format() in QJM (todd)
+
+ HDFS-3795. QJM: validate journal dir at startup (todd)
+
+ HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid
+ segment (todd)
+
+ HDFS-3799. QJM: handle empty log segments during recovery (todd)
+
+ HDFS-3797. QJM: add segment txid as a parameter to journal() RPC (todd)
+
+ HDFS-3800. improvements to QJM fault testing (todd)
+
+ HDFS-3823. QJM: TestQJMWithFaults fails occasionally because of missed
+ setting of HTTP port. (todd and atm)
+
+ HDFS-3826. QJM: Some trivial logging / exception text improvements. (todd
+ and atm)
+
+ HDFS-3839. QJM: hadoop-daemon.sh should be updated to accept "journalnode"
+ (eli)
+
+ HDFS-3845. Fixes for edge cases in QJM recovery protocol (todd)
+
+ HDFS-3877. QJM: Provide defaults for dfs.journalnode.*address (eli)
+
+ HDFS-3863. Track last "committed" txid in QJM (todd)
+
+ HDFS-3869. Expose non-file journal manager details in web UI (todd)
+
+ HDFS-3884. Journal format() should reset cached values (todd)
+
+ HDFS-3870. Add metrics to JournalNode (todd)
+
+ HDFS-3891. Make selectInputStreams throw IOE instead of RTE (todd)
+
+ HDFS-3726. If a logger misses an RPC, don't retry that logger until next
+ segment (todd)
+
+ HDFS-3893. QJM: Make QJM work with security enabled. (atm)
+
+ HDFS-3897. QJM: TestBlockToken fails after HDFS-3893. (atm)
+
+ HDFS-3898. QJM: enable TCP_NODELAY for IPC (todd)
+
+ HDFS-3885. QJM: optimize log sync when JN is lagging behind (todd)
+
+ HDFS-3900. QJM: avoid validating log segments on log rolls (todd)
+
+ HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are
+ out-of-sync (todd)
+
+ HDFS-3899. QJM: Add client-side metrics (todd)
+
+ HDFS-3914. QJM: acceptRecovery should abort current segment (todd)
+
+ HDFS-3915. QJM: Failover fails with auth error in secure cluster (todd)
+
+ HDFS-3906. QJM: quorum timeout on failover with large log segment (todd)
+
+ HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error
+ before being formatted (todd)
+
+ HDFS-3894. QJM: testRecoverAfterDoubleFailures can be flaky due to IPC
+ client caching (todd)
+
+ HDFS-3926. QJM: Add user documentation for QJM. (atm)
+
+ HDFS-3943. QJM: remove currently-unused md5sum field (todd)
+
+ HDFS-3950. QJM: misc TODO cleanup, improved log messages, etc. (todd)
+
+ HDFS-3955. QJM: Make acceptRecovery() atomic. (todd)
+
+ HDFS-3956. QJM: purge temporary files when no longer within retention
+ period (todd)
+
+ HDFS-4004. TestJournalNode#testJournal fails because of test case execution
+ order (Chao Shi via todd)
+
+ HDFS-4017. Unclosed FileInputStream in GetJournalEditServlet
+ (Chao Shi via todd)
+
+ HDFS-4351. In BlockPlacementPolicyDefault.chooseTarget(..), numOfReplicas
+ needs to be updated when avoiding stale nodes. (Andrew Wang via szetszwo)
+
+ HDFS-2908. Add apache license header for StorageReport.java. (Brandon Li
+ via tgraves)
+
+ HDFS-4399. Fix RAT warnings by excluding images sub-dir in docs. (Thomas
+ Graves via acmurthy)
+
+Release 2.0.2-alpha - 2012-09-07
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-3446. HostsFileReader silently ignores bad includes/excludes
+ (Matthew Jacobs via todd)
+
+ HDFS-3755. Creating an already-open-for-write file with overwrite=true fails
+ (todd)
+
+ NEW FEATURES
+
+ HDFS-744. Support hsync in HDFS. (Lars Hofhansl via szetszwo)
+
+ HDFS-3042. Automatic failover support for NameNode HA (todd)
+ (see dedicated section below for breakdown of subtasks)
+
+ HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if
+ the given HDFS is healthy. (szetszwo)
+
+ HDFS-3113. httpfs does not support delegation tokens. (tucu)
+
+ HDFS-3513. HttpFS should cache filesystems. (tucu)
+
+ HDFS-3637. Add support for encrypting the DataTransferProtocol. (atm)
+
+ HDFS-3150. Add option for clients to contact DNs via hostname. (eli)
+
+ HDFS-2793. Add an admin command to trigger an edit log roll. (todd)
+
+ HDFS-3703. Datanodes are marked stale if heartbeat is not received in
+ configured timeout and are selected as the last location to read from.
+ (Jing Zhao via suresh)
+
+ IMPROVEMENTS
+
+ HDFS-3040. TestMulitipleNNDataBlockScanner is misspelled. (Madhukara Phatak
+ via atm)
+
+ HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG
+ logging is enabled. (atm)
+
+ HDFS-3341. Change minimum RPC versions to respective SNAPSHOTs instead of
+ final releases. (todd)
+
+ HDFS-3369. Rename {get|set|add}INode(..) methods in BlockManager and
+ BlocksMap to {get|set|add}BlockCollection(..). (John George via szetszwo)
+
+ HDFS-3134. harden edit log loader against malformed or malicious input.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3230. Cleanup DatanodeID creation in the tests. (eli)
+
+ HDFS-3401. Cleanup DatanodeDescriptor creation in the tests. (eli)
+
+ HDFS-3400. DNs should be able start with jsvc even if security is disabled.
+ (atm via eli)
+
+ HDFS-3404. Make putImage in GetImageServlet infer remote address to fetch
+ from request. (atm)
+
+ HDFS-3335. check for edit log corruption at the end of the log
+ (Colin Patrick McCabe via todd)
+
+ HDFS-3417. Rename BalancerDatanode#getName to getDisplayName to be
+ consistent with Datanode. (eli)
+
+ HDFS-3416. Cleanup DatanodeID and DatanodeRegistration
+ constructors used by testing. (eli)
+
+ HDFS-3419. Cleanup LocatedBlock. (eli)
+
+ HDFS-3440. More effectively limit stream memory consumption when reading
+ corrupt edit logs (Colin Patrick McCabe via todd)
+
+ HDFS-3438. BootstrapStandby should not require a rollEdits on active node
+ (todd)
+
+ HDFS-2885. Remove "federation" from the nameservice config options.
+ (Tsz Wo (Nicholas) Sze via eli)
+
+ HDFS-3394. Do not use generic in INodeFile.getLastBlock(): the run-time
+ ClassCastException check is useless since generic type information is only
+ available in compile-time. (szetszwo)
+
+ HDFS-3454. Balancer unconditionally logs InterruptedException at
+ INFO level on shutdown if security is enabled. (eli)
+
+ HDFS-1013. Miscellaneous improvements to HTML markup for web UIs
+ (Eugene Koontz via todd)
+
+ HDFS-3052. Change INodeFile and INodeFileUnderConstruction to package
+ private. (szetszwo)
+
+ HDFS-3520. Add transfer rate logging to TransferFsImage. (eli)
+
+ HDFS-3504. Support configurable retry policy in DFSClient for RPC
+ connections and RPC calls, and add MultipleLinearRandomRetry, a new retry
+ policy. (szetszwo)
+
+ HDFS-3372. offlineEditsViewer should be able to read a binary
+ edits file with recovery mode. (Colin Patrick McCabe via eli)
+
+ HDFS-3516. Check content-type in WebHdfsFileSystem. (szetszwo)
+
+ HDFS-3535. Audit logging should log denied accesses. (Andy Isaacson via eli)
+
+ HDFS-3481. Refactor HttpFS handling of JAX-RS query string parameters (tucu)
+
+ HDFS-3572. Cleanup code which inits SPNEGO in HttpServer (todd)
+
+ HDFS-3475. Make the replication monitor multipliers configurable.
+ (harsh via eli)
+
+ HDFS-3343. Improve metrics for DN read latency (Andrew Wang via todd)
+
+ HDFS-3170. Add more useful metrics for write latency (Matthew Jacobs via
+ todd)
+
+ HDFS-3604. Add dfs.webhdfs.enabled to hdfs-default.xml. (eli)
+
+ HDFS-2988. Improve error message when storage directory lock fails
+ (Miomir Boljanovic via harsh)
+
+ HDFS-2391. Newly set BalancerBandwidth value is not displayed anywhere.
+ (harsh)
+
+ HDFS-3067. NPE in DFSInputStream.readBuffer if read is repeated on
+ corrupted block. (Henry Robinson via atm)
+
+ HDFS-3555. idle client socket triggers DN ERROR log
+ (should be INFO or DEBUG). (Andy Isaacson via harsh)
+
+ HDFS-3568. fuse_dfs: add support for security. (Colin McCabe via atm)
+
+ HDFS-3629. Fix the typo in the error message about inconsistent
+ storage layout version. (Brandon Li via harsh)
+
+ HDFS-3613. GSet prints some INFO level values, which aren't
+ really very useful to all (Andrew Wang via harsh)
+
+ HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping
+ a few bytes. (Colin Patrick McCabe via harsh)
+
+ HDFS-3582. Hook daemon process exit for testing. (eli)
+
+ HDFS-3641. Move server Util time methods to common and use now
+ instead of System#currentTimeMillis. (eli)
+
+ HDFS-3633. libhdfs: hdfsDelete should pass JNI_FALSE or JNI_TRUE.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-799. libhdfs must call DetachCurrentThread when a thread is destroyed.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3306. fuse_dfs: don't lock release operations.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3612. Single namenode image directory config warning can
+ be improved. (Andy Isaacson via harsh)
+
+ HDFS-3606. libhdfs: create self-contained unit test.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3539. libhdfs code cleanups. (Colin Patrick McCabe via eli)
+
+ HDFS-3610. fuse_dfs: Provide a way to use the default (configured) NN URI.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3663. MiniDFSCluster should capture the code path that led to
+ the first ExitException. (eli)
+
+ HDFS-3659. Add missing @Override to methods across the hadoop-hdfs
+ project. (Brandon Li via harsh)
+
+ HDFS-3537. Move libhdfs and fuse-dfs source to native subdirectories.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3665. Add a test for renaming across file systems via a symlink. (eli)
+
+ HDFS-3666. Plumb more exception messages to terminate. (eli)
+
+ HDFS-3673. libhdfs: fix some compiler warnings. (Colin Patrick McCabe via eli)
+
+ HDFS-3675. libhdfs: follow documented return codes. (Colin Patrick McCabe via eli)
+
+ HDFS-1249. With fuse-dfs, chown which only has owner (or only group)
+ argument fails with Input/output error. (Colin Patrick McCabe via eli)
+
+ HDFS-3583. Convert remaining tests to Junit4. (Andrew Wang via atm)
+
+ HDFS-3711. Manually convert remaining tests to JUnit4. (Andrew Wang via atm)
+
+ HDFS-3650. Use MutableQuantiles to provide latency histograms for various
+ operations. (Andrew Wang via atm)
+
+ HDFS-3667. Add retry support to WebHdfsFileSystem. (szetszwo)
+
+ HDFS-3291. add test that covers HttpFS working w/ a non-HDFS Hadoop
+ filesystem (tucu)
+
+ HDFS-3634. Add self-contained, mavenized fuse_dfs test. (Colin Patrick
+ McCabe via atm)
+
+ HDFS-3190. Simple refactors in existing NN code to assist
+ QuorumJournalManager extension. (todd)
+
+ HDFS-3276. initializeSharedEdits should have a -nonInteractive flag (todd)
+
+ HDFS-3765. namenode -initializeSharedEdits should be able to initialize
+ all shared storages. (Vinay and todd via todd)
+
+ HDFS-3723. Add support -h, -help to all the commands. (Jing Zhao via
+ suresh)
+
+ HDFS-3803. Change BlockPoolSliceScanner chatty INFO log to DEBUG.
+ (Andrew Purtell via suresh)
+
+ HDFS-3802. StartupOption.name in HdfsServerConstants should be final.
+ (Jing Zhao via szetszwo)
+
+ HDFS-3796. Speed up edit log tests by avoiding fsync() (todd)
+
+ HDFS-2963. Console Output is confusing while executing metasave
+ (dfsadmin command). (Andrew Wang via eli)
+
+ HDFS-3672. Expose disk-location information for blocks to enable better
+ scheduling. (Andrew Wang via atm)
+
+ HDFS-2727. libhdfs should get the default block size from the server.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3832. Remove protocol methods related to DistributedUpgrade. (suresh)
+
+ HDFS-3819. Should check whether invalidate work percentage default value is
+ not greater than 1.0f. (Jing Zhao via jitendra)
+
+ HDFS-3177. Update DFSClient and DataXceiver to handle different checkum
+ types in file checksum computation. (Kihwal Lee via szetszwo)
+
+ HDFS-3844. Add @Override and remove {@inheritdoc} and unnecessary
+ imports. (Jing Zhao via suresh)
+
+ HDFS-3853. Port MiniDFSCluster enableManagedDfsDirsRedundancy
+ option to branch-2. (Colin Patrick McCabe via eli)
+
+ HDFS-3871. Change NameNodeProxies to use RetryUtils. (Arun C Murthy
+ via szetszwo)
+
+ HDFS-3887. Remove redundant chooseTarget methods in BlockPlacementPolicy.
+ (Jing Zhao via szetszwo)
+
+ HDFS-3888. Clean up BlockPlacementPolicyDefault. (Jing Zhao via szetszwo)
+
+ HDFS-3907. Allow multiple users for local block readers. (eli)
+
+ HDFS-3510. Editlog pre-allocation is performed prior to writing edits
+ to avoid partial edits case disk out of space. (Colin McCabe via todd)
+
+ HDFS-3910. DFSTestUtil#waitReplication should timeout. (eli)
+
+ HDFS-3920. libwebdhfs string processing and using strerror consistently
+ to handle all errors. (Jing Zhao via suresh)
+
+ OPTIMIZATIONS
+
+ HDFS-2982. Startup performance suffers when there are many edit log
+ segments. (Colin Patrick McCabe via todd)
+
+ HDFS-2834. Add a ByteBuffer-based read API to DFSInputStream.
+ (Henry Robinson via todd)
+
+ HDFS-3110. Use directRead API to reduce the number of buffer copies in
+ libhdfs (Henry Robinson via todd)
+
+ HDFS-3697. Enable fadvise readahead by default. (todd)
+
+ HDFS-2421. Improve the concurrency of SerialNumberMap in NameNode.
+ (Jing Zhao and Weiyan Wang via szetszwo)
+
+ HDFS-3866. HttpFS POM should have property where to download tomcat from (zero45 via tucu)
+
+ BUG FIXES
+
+ HDFS-3385. The last block of INodeFileUnderConstruction is not
+ necessarily a BlockInfoUnderConstruction, so do not cast it in
+ FSNamesystem.recoverLeaseInternal(..). (szetszwo)
+
+ HDFS-3414. Balancer does not find NameNode if rpc-address or
+ servicerpc-address are not set in client configs. (atm)
+
+ HDFS-3031. Fix complete() and getAdditionalBlock() RPCs to be idempotent
+ (todd)
+
+ HDFS-2759. Pre-allocate HDFS edit log files after writing version number.
+ (atm)
+
+ HDFS-3413. TestFailureToReadEdits timing out. (atm)
+
+ HDFS-3422. TestStandbyIsHot timeouts too aggressive (todd)
+
+ HDFS-3433. GetImageServlet should allow administrative requestors when
+ security is enabled. (atm)
+
+ HDFS-1153. dfsnodelist.jsp should handle invalid input parameters.
+ (Ravi Phulari via eli)
+
+ HDFS-3434. InvalidProtocolBufferException when visiting DN
+ browseDirectory.jsp (eli)
+
+ HDFS-2800. Fix cancellation of checkpoints in the standby node to be more
+ reliable. (todd)
+
+ HDFS-3391. Fix InvalidateBlocks to compare blocks including their
+ generation stamps. (todd)
+
+ HDFS-3444. hdfs groups command doesn't work with security enabled. (atm)
+
+ HDFS-2717. BookKeeper Journal output stream doesn't check addComplete rc.
+ (Ivan Kelly via umamahesh)
+
+ HDFS-3415. Make sure all layout versions are the same for all storage
+ directories in the Namenode. (Brandon Li via szetszwo)
+
+ HDFS-3436. In DataNode.transferReplicaForPipelineRecovery(..), it should
+ use the stored generation stamp to check if the block is valid. (Vinay
+ via szetszwo)
+
+ HDFS-3460. HttpFS proxyuser validation with Kerberos ON uses full
+ principal name. (tucu)
+
+ HDFS-3058. HA: Bring BookKeeperJournalManager up to date with HA changes.
+ (Ivan Kelly via umamahesh)
+
+ HDFS-3368. Missing blocks due to bad DataNodes coming up and down. (shv)
+
+ HDFS-3452. BKJM:Switch from standby to active fails and NN gets shut down
+ due to delay in clearing of lock. (umamahesh)
+
+ HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline.
+ (Amith D K via umamahesh)
+
+ HDFS-3474. Cleanup Exception handling in BookKeeper journal manager.
+ (Ivan Kelly via umamahesh)
+
+ HDFS-3468. Make BKJM-ZK session timeout configurable. (umamahesh)
+
+ HDFS-3423. BKJM: NN startup is failing, when tries to recoverUnfinalizedSegments()
+ a bad inProgress_ ZNodes. (Ivan Kelly and Uma via umamahesh)
+
+ HDFS-3441. Race condition between rolling logs at active NN and purging at standby.
+ (Rakesh R via umamahesh)
+
+ HDFS-3484. hdfs fsck doesn't work if NN HTTP address is set to
+ 0.0.0.0 even if NN RPC address is configured. (atm via eli)
+
+ HDFS-3486. offlineimageviewer can't read fsimage files that contain
+ persistent delegation tokens. (Colin Patrick McCabe via eli)
+
+ HDFS-3487. offlineimageviewer should give byte offset information
+ when it encounters an exception. (Colin Patrick McCabe via eli)
+
+ HDFS-3442. Incorrect count for Missing Replicas in FSCK report. (Andrew
+ Wang via atm)
+
+ HDFS-2025. Go Back to File View link is not working in tail.jsp.
+ (Ashish and Sravan via umamahesh)
+
+ HDFS-3501. Checkpointing with security enabled will stop working
+ after ticket lifetime expires. (atm via eli)
+
+ HDFS-3266. DFSTestUtil#waitCorruptReplicas doesn't sleep between checks.
+ (Madhukara Phatak via atm)
+
+ HDFS-3505. DirectoryScanner does not join all threads in shutdown.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3485. DataTransferThrottler will over-throttle when currentTimeMillis
+ jumps (Andy Isaacson via todd)
+
+ HDFS-2914. HA: Standby should not enter safemode when resources are low.
+ (Vinay via atm)
+
+ HDFS-3235. MiniDFSClusterManager doesn't correctly support -format option.
+ (Henry Robinson via atm)
+
+ HDFS-3514. Add missing TestParallelLocalRead. (Henry Robinson via atm)
+
+ HDFS-3243. TestParallelRead timing out on jenkins. (Henry Robinson via todd)
+
+ HDFS-3490. DatanodeWebHdfsMethods throws NullPointerException if
+ NamenodeRpcAddressParam is not set. (szetszwo)
+
+ HDFS-2797. Fix misuses of InputStream#skip in the edit log code.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3517. TestStartup should bind ephemeral ports. (eli)
+
+ HDFS-3522. If a namenode is in safemode, it should throw SafeModeException
+ when getBlockLocations has zero locations. (Brandon Li via szetszwo)
+
+ HDFS-3408. BKJM : Namenode format fails, if there is no BK root. (Rakesh R via umamahesh)
+
+ HDFS-3389. Document the BKJM usage in Namenode HA. (umamahesh and Ivan Kelly via umamahesh)
+
+ HDFS-3531. EditLogFileOutputStream#preallocate should check for
+ incomplete writes. (Colin Patrick McCabe via eli)
+
+ HDFS-766. Error message not clear for set space quota out of boundary
+ values. (Jon Zuanich via atm)
+
+ HDFS-3480. Multiple SLF4J binding warning. (Vinay via eli)
+
+ HDFS-3524. Update TestFileLengthOnClusterRestart for HDFS-3522. (Brandon
+ Li via szetszwo)
+
+ HDFS-3559. DFSTestUtil: use Builder class to construct DFSTestUtil
+ instances. (Colin Patrick McCabe via atm)
+
+ HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection.
+ (szetszwo)
+
+ HDFS-3157. Fix a bug in the case that the generation stamps of the stored
+ block in a namenode and the reported block from a datanode do not match.
+ (Ashish Singhi via szetszwo)
+
+ HDFS-3575. HttpFS does not log Exception Stacktraces (brocknoland via tucu)
+
+ HDFS-3574. Fix small race and do some cleanup in GetImageServlet (todd)
+
+ HDFS-3581. FSPermissionChecker#checkPermission sticky bit check
+ missing range check. (eli)
+
+ HDFS-3541. Deadlock between recovery, xceiver and packet responder.
+ (Vinay via umamahesh)
+
+ HDFS-3428. Move DelegationTokenRenewer to common (tucu)
+
+ HDFS-3491. HttpFs does not set permissions correctly (tucu)
+
+ HDFS-3580. incompatible types; no instance(s) of type variable(s) V exist
+ so that V conforms to boolean compiling HttpFSServer.java with OpenJDK
+ (adi2 via tucu)
+
+ HDFS-3603. Decouple TestHDFSTrash from TestTrash. (Jason Lowe via eli)
+
+ HDFS-711. hdfsUtime does not handle atime = 0 or mtime = 0 correctly.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3548. NamenodeFsck.copyBlock fails to create a Block Reader.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3615. Two BlockTokenSecretManager findbugs warnings. (atm)
+
+ HDFS-470. libhdfs should handle 0-length reads from FSInputStream
+ correctly. (Colin Patrick McCabe via eli)
+
+ HDFS-3492. fix some misuses of InputStream#skip.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3609. libhdfs: don't force the URI to look like hdfs://hostname:port.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-2966 TestNameNodeMetrics tests can fail under load. (stevel)
+
+ HDFS-3605. Block mistakenly marked corrupt during edit log catchup
+ phase of failover. (todd and Brahma Reddy Battula via todd)
+
+ HDFS-3690. BlockPlacementPolicyDefault incorrectly casts LOG. (eli)
+
+ HDFS-3597. SNN fails to start after DFS upgrade. (Andy Isaacson via todd)
+
+ HDFS-3608. fuse_dfs: detect changes in UID ticket cache. (Colin Patrick
+ McCabe via atm)
+
+ HDFS-3709. TestStartup tests still binding to the ephemeral port. (eli)
+
+ HDFS-3720. hdfs.h must get packaged. (Colin Patrick McCabe via atm)
+
+ HDFS-3626. Creating file with invalid path can corrupt edit log (todd)
+
+ HDFS-3679. fuse_dfs notrash option sets usetrash. (Conrad Meyer via suresh)
+
+ HDFS-3732. fuse_dfs: incorrect configuration value checked for connection
+ expiry timer period. (Colin Patrick McCabe via atm)
+
+ HDFS-3738. TestDFSClientRetries#testFailuresArePerOperation sets incorrect
+ timeout config. (atm)
+
+ HDFS-3756. DelegationTokenFetcher creates 2 HTTP connections, the second
+ one not properly configured. (tucu)
+
+ HDFS-3579. libhdfs: fix exception handling. (Colin Patrick McCabe via atm)
+
+ HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. (eli)
+
+ HDFS-3760. primitiveCreate is a write, not a read. (Andy Isaacson via atm)
+
+ HDFS-3710. libhdfs misuses O_RDONLY/WRONLY/RDWR. (Andy Isaacson via atm)
+
+ HDFS-3721. hsync support broke wire compatibility. (todd and atm)
+
+ HDFS-3758. TestFuseDFS test failing. (Colin Patrick McCabe via eli)
+
+ HDFS-2330. In NNStorage.java, IOExceptions of stream closures can mask
+ root exceptions. (umamahesh via todd)
+
+ HDFS-3790. test_fuse_dfs.c doesn't compile on centos 5. (Colin Patrick
+ McCabe via atm)
+
+ HDFS-3658. Fix bugs in TestDFSClientRetries and add more tests. (szetszwo)
+
+ HDFS-3794. WebHDFS OPEN returns the incorrect Content-Length in the HTTP
+ header when offset is specified and length is omitted.
+ (Ravi Prakash via szetszwo)
+
+ HDFS-3048. Small race in BlockManager#close. (Andy Isaacson via eli)
+
+ HDFS-3194. DataNode block scanner is running too frequently.
+ (Andy Isaacson via eli)
+
+ HDFS-3808. fuse_dfs: postpone libhdfs intialization until after fork.
+ (Colin Patrick McCabe via atm)
+
+ HDFS-3788. ByteRangeInputStream should not expect HTTP Content-Length header
+ when chunked transfer-encoding is used. (szetszwo)
+
+ HDFS-3816. Invalidate work percentage default value should be 0.32f
+ instead of 32. (Jing Zhao via suresh)
+
+ HDFS-3707. TestFSInputChecker: improper use of skip.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3830. test_libhdfs_threaded: use forceNewInstance.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3835. Long-lived 2NN cannot perform a checkpoint if security is
+ enabled and the NN restarts with outstanding delegation tokens. (atm)
+
+ HDFS-3715. Fix TestFileCreation#testFileCreationNamenodeRestart.
+ (Andrew Whang via eli)
+
+ HDFS-3683. Edit log replay progress indicator shows >100% complete. (Plamen
+ Jeliazkov via atm)
+
+ HDFS-3731. Release upgrade must handle blocks being written from 1.0.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3856. TestHDFSServerPorts failure is causing surefire fork failure.
+ (eli)
+
+ HDFS-3860. HeartbeatManager#Monitor may wrongly hold the writelock of
+ namesystem. (Jing Zhao via atm)
+
+ HDFS-3849. When re-loading the FSImage, we should clear the existing
+ genStamp and leases. (Colin Patrick McCabe via atm)
+
+ HDFS-3864. NN does not update internal file mtime for OP_CLOSE when reading
+ from the edit log. (atm)
+
+ HDFS-3837. Fix DataNode.recoverBlock findbugs warning. (eli)
+
+ HDFS-3733. Audit logs should include WebHDFS access. (Andy Isaacson via
+ eli)
+
+ HDFS-2686. Remove DistributedUpgrade related code. (suresh)
+
+ HDFS-3833. TestDFSShell fails on windows due to concurrent file
+ read/write. (Brandon Li via suresh)
+
+ HDFS-3466. Get HTTP kerberos principal from the web authentication keytab.
+ (omalley)
+
+ HDFS-3879. Fix findbugs warning in TransferFsImage on branch-2. (eli)
+
+ HDFS-3469. start-dfs.sh will start zkfc, but stop-dfs.sh will not stop zkfc similarly.
+ (Vinay via umamahesh)
+
+ HDFS-1490. TransferFSImage should timeout (Dmytro Molkov and Vinay via todd)
+
+ HDFS-3828. Block Scanner rescans blocks too frequently.
+ (Andy Isaacson via eli)
+
+ HDFS-3895. hadoop-client must include commons-cli (tucu)
+
+ HDFS-2757. Cannot read a local block that's being written to when
+ using the local read short circuit. (Jean-Daniel Cryans via eli)
+
+ HDFS-3664. BlockManager race when stopping active services.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3928. MiniDFSCluster should reset the first ExitException on shutdown. (eli)
+
+ HDFS-3938. remove current limitations from HttpFS docs. (tucu)
+
+ HDFS-3944. Httpfs resolveAuthority() is not resolving host correctly. (tucu)
+
+ HDFS-3972. Trash emptier fails in secure HA cluster. (todd via eli)
+
+ HDFS-4443. Remove a trailing '`' character from the HTML code generated by
+ NamenodeJspHelper.generateNodeData(..). (Christian Rohling via szetszwo)
+
+ BREAKDOWN OF HDFS-3042 SUBTASKS
+
+ HDFS-2185. HDFS portion of ZK-based FailoverController (todd)
+
+ HDFS-3200. Scope all ZKFC configurations by nameservice (todd)
+
+ HDFS-3223. add zkfc to hadoop-daemon.sh script (todd)
+
+ HDFS-3261. TestHASafeMode fails on HDFS-3042 branch (todd)
+
+ HDFS-3159. Document NN auto-failover setup and configuration (todd)
+
+ HDFS-3412. Fix findbugs warnings in auto-HA branch (todd)
+
+ HDFS-3432. TestDFSZKFailoverController tries to fail over too early (todd)
+
+ HDFS-3902. TestDatanodeBlockScanner#testBlockCorruptionPolicy is broken.
+ (Andy Isaacson via eli)
+
+Release 2.0.0-alpha - 05-23-2012
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-2676. Remove Avro RPC. (suresh)
+
+ HDFS-2303. Unbundle jsvc. (Roman Shaposhnik and Mingjie Lai via eli)
+
+ HDFS-3137. Bump LAST_UPGRADABLE_LAYOUT_VERSION to -16. (eli)
+
+ HDFS-3138. Move DatanodeInfo#ipcPort to DatanodeID. (eli)
+
+ HDFS-3164. Move DatanodeInfo#hostName to DatanodeID. (eli)
+
+ NEW FEATURES
+
+ HDFS-2978. The NameNode should expose name dir statuses via JMX. (atm)
+
+ HDFS-395. DFS Scalability: Incremental block reports. (Tomasz Nykiel
+ via hairong)
+
+ HDFS-2517. Add protobuf service for JounralProtocol. (suresh)
+
+ HDFS-2518. Add protobuf service for NamenodeProtocol. (suresh)
+
+ HDFS-2520. Add protobuf service for InterDatanodeProtocol. (suresh)
+
+ HDFS-2519. Add protobuf service for DatanodeProtocol. (suresh)
+
+ HDFS-2581. Implement protobuf service for JournalProtocol. (suresh)
+
+ HDFS-2618. Implement protobuf service for NamenodeProtocol. (suresh)
+
+ HDFS-2629. Implement protobuf service for InterDatanodeProtocol. (suresh)
+
+ HDFS-2636. Implement protobuf service for ClientDatanodeProtocol. (suresh)
+
+ HDFS-2642. Protobuf translators for DatanodeProtocol. (jitendra)
+
+ HDFS-2647. Used protobuf based RPC for InterDatanodeProtocol,
+ ClientDatanodeProtocol, JournalProtocol, NamenodeProtocol. (suresh)
+
+ HDFS-2661. Enable protobuf RPC for DatanodeProtocol. (jitendra)
+
+ HDFS-2697. Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings
+ protocol to protocol buffers. (jitendra)
+
+ HDFS-2880. Protobuf changes in DatanodeProtocol to add multiple storages.
+ (suresh)
+
+ HDFS-2899. Service protocol changes in DatanodeProtocol to add multiple
+ storages. (suresh)
+
+ HDFS-2430. The number of failed or low-resource volumes the NN can tolerate
+ should be configurable. (atm)
+
+ HDFS-1623. High Availability Framework for HDFS NN. Contributed by Todd
+ Lipcon, Aaron T. Myers, Eli Collins, Uma Maheswara Rao G, Bikas Saha,
+ Suresh Srinivas, Jitendra Nath Pandey, Hari Mankude, Brandon Li, Sanjay
+ Radia, Mingjie Lai, and Gregory Chanan
+
+ HDFS-2941. Add an administrative command to download a copy of the fsimage
+ from the NN. (atm)
+
+ HDFS-2413. Add an API DistributedFileSystem.isInSafeMode() and change
+ DistributedFileSystem to @InterfaceAudience.LimitedPrivate.
+ (harsh via szetszwo)
+
+ HDFS-3167. CLI-based driver for MiniDFSCluster. (Henry Robinson via atm)
+
+ HDFS-3148. The client should be able to use multiple local interfaces
+ for data transfer. (eli)
+
+ HDFS-3000. Add a public API for setting quotas. (atm)
+
+ HDFS-3102. Add CLI tool to initialize the shared-edits dir. (atm)
+
+ HDFS-3004. Implement Recovery Mode. (Colin Patrick McCabe via eli)
+
+ HDFS-3282. Add HdfsDataInputStream as a public API. (umamahesh)
+
+ HDFS-3298. Add HdfsDataOutputStream as a public API. (szetszwo)
+
+ HDFS-234. Integration with BookKeeper logging system. (Ivan Kelly
+ via umamahesh)
+
+ IMPROVEMENTS
+
+ HDFS-2018. Move all journal stream management code into one place.
+ (Ivan Kelly via jitendra)
+
+ HDFS-2223. Untangle depencencies between NN components (todd)
+
+ HDFS-2351. Change Namenode and Datanode to register each of their protocols
+ seperately (sanjay)
+
+ HDFS-2337. DFSClient shouldn't keep multiple RPC proxy references (atm)
+
+ HDFS-2181. Separate HDFS Client wire protocol data types (sanjay)
+
+ HDFS-2459. Separate datatypes for Journal Protocol. (suresh)
+
+ HDFS-2480. Separate datatypes for NamenodeProtocol. (suresh)
+
+ HDFS-2489. Move Finalize and Register to separate file out of
+ DatanodeCommand.java. (suresh)
+
+ HDFS-2488. Separate datatypes for InterDatanodeProtocol. (suresh)
+
+ HDFS-2496. Separate datatypes for DatanodeProtocol. (suresh)
+
+ HDFS-2479. HDFS Client Data Types in Protocol Buffers (sanjay)
+
+ HADOOP-7862. Hdfs changes to work with HADOOP-7862: Move the support for
+ multiple protocols to lower layer so that Writable, PB and Avro can all
+ use it. (sanjay)
+
+ HDFS-2597. ClientNameNodeProtocol in Protocol Buffers. (sanjay)
+
+ HDFS-2651. ClientNameNodeProtocol Translators for Protocol Buffers. (sanjay)
+
+ HDFS-2650. Replace @inheritDoc with @Override. (Hari Mankude via suresh).
+
+ HDFS-2669. Enable protobuf rpc for ClientNamenodeProtocol. (sanjay)
+
+ HDFS-2801. Provide a method in client side translators to check for a
+ methods supported in underlying protocol. (jitendra)
+
+ HDFS-2895. Remove Writable wire protocol types and translators to
+ complete transition to protocol buffers. (suresh)
+
+ HDFS-2992. Edit log failure trace should include transaction ID of
+ error. (Colin Patrick McCabe via eli)
+
+ HDFS-3030. Remove getProtocolVersion and getProtocolSignature from
+ translators. (jitendra)
+
+ HDFS-2158. Add JournalSet to manage the set of journals. (jitendra)
+
+ HDFS-2334. Add Closeable to JournalManager. (Ivan Kelly via jitendra)
+
+ HDFS-1580. Add interface for generic Write Ahead Logging mechanisms.
+ (Ivan Kelly via jitendra)
+
+ HDFS-3060. Bump TestDistributedUpgrade#testDistributedUpgrade timeout (eli)
+
+ HDFS-2410. Further cleanup of hardcoded configuration keys and values.
+ (suresh)
+
+ HDFS-2878. Fix TestBlockRecovery and move it back into main test directory.
+ (todd)
+
+ HDFS-3003. Remove getHostPortString() from NameNode, replace it with
+ NetUtils.getHostPortString(). (Brandon Li via atm)
+
+ HDFS-3014. FSEditLogOp and its subclasses should have toString() method.
+ (Sho Shimauchi via atm)
+
+ HDFS-3021. Use generic type to declare FSDatasetInterface. (szetszwo)
+
+ HDFS-3056. Add a new interface RollingLogs for DataBlockScanner logging.
+ (szetszwo)
+
+ HDFS-2731. Add command to bootstrap the Standby Node's name directories
+ from the Active NameNode. (todd)
+
+ HDFS-3082. Clean up FSDatasetInterface and change DataNode.data to package
+ private. (szetszwo)
+
+ HDFS-3057. httpfs and hdfs launcher scripts should honor CATALINA_HOME
+ and HADOOP_LIBEXEC_DIR (rvs via tucu)
+
+ HDFS-3088. Move FSDatasetInterface inner classes to a package. (szetszwo)
+
+ HDFS-3111. Missing license headers in trunk. (umamahesh)
+
+ HDFS-3091. Update the usage limitations of ReplaceDatanodeOnFailure policy in
+ the config description for the smaller clusters. (szetszwo via umamahesh)
+
+ HDFS-3105. Add DatanodeStorage information to block recovery. (szetszwo)
+
+ HDFS-3086. Change Datanode not to send storage list in registration.
+ (szetszwo)
+
+ HDFS-309. FSEditLog should log progress during replay. (Sho Shimauchi
+ via todd)
+
+ HDFS-3044. fsck move should be non-destructive by default.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3071. haadmin failover command does not provide enough detail when
+ target NN is not ready to be active. (todd)
+
+ HDFS-3089. Move FSDatasetInterface and the related classes to a package.
+ (szetszwo)
+
+ HDFS-3129. NetworkTopology: add test that getLeaf should check for
+ invalid topologies. (Colin Patrick McCabe via eli)
+
+ HDFS-3155. Clean up FSDataset implemenation related code. (szetszwo)
+
+ HDFS-3158. LiveNodes member of NameNodeMXBean should list non-DFS used
+ space and capacity per DN. (atm)
+
+ HDFS-3172. dfs.upgrade.permission is dead code. (eli)
+
+ HDFS-3171. The DatanodeID "name" field is overloaded. (eli)
+
+ HDFS-3144. Refactor DatanodeID#getName by use. (eli)
+
+ HDFS-3130. Move fsdataset implementation to a package. (szetszwo)
+
+ HDFS-3120. Enable hsync and hflush by default. (eli)
+
+ HDFS-3187. Upgrade guava to 11.0.2 (todd)
+
+ HDFS-3168. Remove unnecessary "throw IOException" and change fields to
+ final in FSNamesystem and BlockManager. (szetszwo)
+
+ HDFS-2564. Cleanup unnecessary exceptions thrown and unnecessary casts.
+ (Hari Mankude via eli)
+
+ HDFS-3084. FenceMethod.tryFence() and ShellCommandFencer should pass
+ namenodeId as well as host:port (todd)
+
+ HDFS-3050. rework OEV to share more code with the NameNode.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3204. Minor modification to JournalProtocol.proto to make
+ it generic. (suresh)
+
+ HDFS-3226. Allow GetConf tool to print arbitrary keys (todd)
+
+ HDFS-3240. Drop log level of "heartbeat: ..." in BPServiceActor to DEBUG
+ (todd)
+
+ HDFS-3238. ServerCommand and friends don't need to be writables. (eli)
+
+ HDFS-3094. add -nonInteractive and -force option to namenode -format
+ command (Arpit Gupta via todd)
+
+ HDFS-3244. Remove dead writable code from hdfs/protocol. (eli)
+
+ HDFS-3247. Improve bootstrapStandby behavior when original NN is not active
+ (todd)
+
+ HDFS-3249. Use ToolRunner.confirmPrompt in NameNode (todd)
+
+ HDFS-3179. Improve the exception message thrown by DataStreamer when
+ it failed to add a datanode. (szetszwo)
+
+ HDFS-2983. Relax the build version check to permit rolling upgrades within a release. (atm)
+
+ HDFS-3259. NameNode#initializeSharedEdits should populate shared edits dir
+ with edit log segments. (atm)
+
+ HDFS-2708. Stats for the # of blocks per DN. (atm)
+
+ HDFS-3279. Move the FSEditLog constructor with @VisibleForTesting to
+ TestEditLog. (Arpit Gupta via szetszwo)
+
+ HDFS-3294. Fix code indentation in NamenodeWebHdfsMethods and
+ DatanodeWebHdfsMethods. (szetszwo)
+
+ HDFS-3263. HttpFS should read HDFS config from Hadoop site.xml files (tucu)
+
+ HDFS-3206. Miscellaneous xml cleanups for OEV.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3169. TestFsck should test multiple -move operations in a row.
+ (Colin Patrick McCabe via eli)
+
+ HDFS-3258. Test for HADOOP-8144 (pseudoSortByDistance in
+ NetworkTopology for first rack local node). (Junping Du via eli)
+
+ HDFS-3322. Use HdfsDataInputStream and HdfsDataOutputStream in Hdfs.
+ (szetszwo)
+
+ HDFS-3339. Change INode to package private. (John George via szetszwo)
+
+ HDFS-3303. Remove Writable implementation from RemoteEditLogManifest.
+ (Brandon Li via szetszwo)
+
+ HDFS-2617. Replaced Kerberized SSL for image transfer and fsck
+ with SPNEGO-based solution. (jghoman, omalley, tucu, and atm via eli)
+
+ HDFS-3365. Enable users to disable socket caching in DFS client
+ configuration (todd)
+
+ HDFS-3375. Put client name in DataXceiver thread name for readBlock
+ and keepalive (todd)
+
+ HDFS-3363. Define BlockCollection and MutableBlockCollection interfaces
+ so that INodeFile and INodeFileUnderConstruction do not have to be used in
+ block management. (John George via szetszwo)
+
+ HDFS-3211. Add fence(..) and replace NamenodeRegistration with JournalInfo
+ and epoch in JournalProtocol. (suresh via szetszwo)
+
+ HADOOP-8285 HDFS changes for Use ProtoBuf for RpcPayLoadHeader (sanjay radia)
+
+ HDFS-3418. Rename BlockWithLocationsProto datanodeIDs field to storageIDs.
+ (eli)
+
+ OPTIMIZATIONS
+
+ HDFS-2477. Optimize computing the diff between a block report and the
+ namenode state. (Tomasz Nykiel via hairong)
+
+ HDFS-2495. Increase granularity of write operations in ReplicationMonitor
+ thus reducing contention for write lock. (Tomasz Nykiel via hairong)
+
+ HDFS-2476. More CPU efficient data structure for under-replicated,
+ over-replicated, and invalidated blocks. (Tomasz Nykiel via todd)
+
+ HDFS-3036. Remove unused method DFSUtil#isDefaultNamenodeAddress. (atm)
+
+ HDFS-3378. Remove DFS_NAMENODE_SECONDARY_HTTPS_PORT_KEY and DEFAULT. (eli)
+
+ BUG FIXES
+
+ HDFS-2481. Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol.
+ (sanjay)
+
+ HDFS-2497. Fix TestBackupNode failure. (suresh)
+
+ HDFS-2499. RPC client is created incorrectly introduced in HDFS-2459.
+ (suresh)
+
+ HDFS-2526. (Client)NamenodeProtocolTranslatorR23 do not need to keep a
+ reference to rpcProxyWithoutRetry (atm)
+
+ HDFS-2532. TestDfsOverAvroRpc timing out in trunk (Uma Maheswara Rao G
+ via todd)
+
+ HDFS-2666. Fix TestBackupNode failure. (suresh)
+
+ HDFS-2663. Optional protobuf parameters are not handled correctly. (suresh)
+
+ HDFS-2694. Removal of Avro broke non-PB NN services. (atm)
+
+ HDFS-2687. Tests failing with ClassCastException post protobuf RPC
+ changes. (suresh)
+
+ HDFS-2700. Fix failing TestDataNodeMultipleRegistrations in trunk
+ (Uma Maheswara Rao G via todd)
+
+ HDFS-2739. SecondaryNameNode doesn't start up. (jitendra)
+
+ HDFS-2768. BackupNode stop can not close proxy connections because
+ it is not a proxy instance. (Uma Maheswara Rao G via eli)
+
+ HDFS-2968. Protocol translator for BlockRecoveryCommand broken when
+ multiple blocks need recovery. (todd)
+
+ HDFS-3020. Fix editlog to automatically sync when buffer is full. (todd)
+
+ HDFS-3038. Add FSEditLog.metrics to findbugs exclude list. (todd via atm)
+
+ HDFS-2188. Make FSEditLog create its journals from a list of URIs rather
+ than NNStorage. (Ivan Kelly via jitendra)
+
+ HDFS-1765. Block Replication should respect under-replication
+ block priority. (Uma Maheswara Rao G via eli)
+
+ HDFS-2285. BackupNode should reject requests to modify namespace.
+ (shv and Uma Maheswara Rao)
+
+ HDFS-2764. TestBackupNode is racy. (atm)
+
+ HDFS-3093. Fix bug where namenode -format interpreted the -force flag in
+ reverse. (todd)
+
+ HDFS-3005. FSVolume.decDfsUsed(..) should be synchronized. (szetszwo)
+
+ HDFS-3099. SecondaryNameNode does not properly initialize metrics system.
+ (atm)
+
+ HDFS-3062. Fix bug which prevented MR job submission from creating
+ delegation tokens on an HA cluster. (Mingjie Lai via todd)
+
+ HDFS-3083. Cannot run an MR job with HA and security enabled when
+ second-listed NN active. (atm)
+
+ HDFS-3100. In BlockSender, throw an exception when it needs to verify
+ checksum but the meta data does not exist. (Brandon Li via szetszwo)
+
+ HDFS-3132. Fix findbugs warning on HDFS trunk. (todd)
+
+ HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. (atm)
+
+ HDFS-3143. TestGetBlocks.testGetBlocks is failing. (Arpit Gupta via atm)
+
+ HDFS-3142. TestHDFSCLI.testAll is failing. (Brandon Li via atm)
+
+ HDFS-3070. HDFS balancer doesn't ensure that hdfs-site.xml is loaded. (atm)
+
+ HDFS-2995. start-dfs.sh should only start the 2NN for namenodes
+ with dfs.namenode.secondary.http-address configured. (eli)
+
+ HDFS-3174. Fix assert in TestPendingDataNodeMessages. (eli)
+
+ HDFS-3199. TestValidateConfigurationSettings is failing. (todd via eli)
+
+ HDFS-3202. NamespaceInfo PB translation drops build version. (atm)
+
+ HDFS-3109. Remove hsqldf exclusions from pom.xml. (Ravi Prakash
+ via suresh)
+
+ HDFS-3210. JsonUtil#toJsonMap for for a DatanodeInfo should use
+ "ipAddr" instead of "name". (eli)
+
+ HDFS-3208. Bogus entries in hosts files are incorrectly displayed
+ in the report. (eli)
+
+ HDFS-3136. Remove SLF4J dependency as HDFS does not need it to fix
+ unnecessary warnings. (Jason Lowe via suresh)
+
+ HDFS-3214. InterDatanodeProtocolServerSideTranslatorPB doesn't handle
+ null response from initReplicaRecovery (todd)
+
+ HDFS-3119. Overreplicated block is not deleted even after the replication
+ factor is reduced after sync follwed by closing that file. (Ashish Singhi
+ via umamahesh)
+
+ HDFS-3234. Accidentally left log message in GetConf after HDFS-3226 (todd)
+
+ HDFS-3236. NameNode does not initialize generic conf keys when started
+ with -initializeSharedEditsDir (atm)
+
+ HDFS-3248. bootstrapStandby repeated twice in hdfs namenode usage message
+ (Colin Patrick McCabe via todd)
+
+ HDFS-2696. Fix the fuse-fds build. (Bruno Mahé via eli)
+
+ HDFS-3254. Branch-2 build broken due to wrong version number in
+ fuse-dfs' pom.xml. (Anupam Seth via eli)
+
+ HDFS-3260. TestDatanodeRegistration should set minimum DN version in
+ addition to minimum NN version. (atm)
+
+ HDFS-3255. HA DFS returns wrong token service (Daryn Sharp via todd)
+
+ HDFS-3256. HDFS considers blocks under-replicated if topology script is
+ configured with only 1 rack. (atm)
+
+ HDFS-2799. Trim fs.checkpoint.dir values. (Amith D K via eli)
+
+ HDFS-2765. TestNameEditsConfigs is incorrectly swallowing IOE. (atm)
+
+ HDFS-3280. DFSOutputStream.sync should not be synchronized (todd)
+
+ HDFS-3268. FileContext API mishandles token service and incompatible with
+ HA (Daryn Sharp via todd)
+
+ HDFS-3284. bootstrapStandby fails in secure cluster (todd)
+
+ HDFS-3165. HDFS Balancer scripts are refering to wrong path of
+ hadoop-daemon.sh (Amith D K via eli)
+
+ HDFS-891. DataNode no longer needs to check for dfs.network.script.
+ (harsh via eli)
+
+ HDFS-3305. GetImageServlet should consider SBN a valid requestor in a
+ secure HA setup. (atm)
+
+ HDFS-3314. HttpFS operation for getHomeDirectory is incorrect. (tucu)
+
+ HDFS-3319. Change DFSOutputStream to not to start a thread in constructors.
+ (szetszwo)
+
+ HDFS-3222. DFSInputStream#openInfo should not silently get the length as 0
+ when locations length is zero for last partial block. (umamahesh)
+
+ HDFS-3181. Fix a test case in TestLeaseRecovery2. (szetszwo)
+
+ HDFS-3309. HttpFS (Hoop) chmod not supporting octal and sticky bit
+ permissions. (tucu)
+
+ HDFS-3326. Append enabled log message uses the wrong variable.
+ (Matthew Jacobs via eli)
+
+ HDFS-3275. Skip format for non-file based directories.
+ (Amith D K via umamahesh)
+
+ HDFS-3286. When the threshold value for balancer is zero, unexpected output is displayed.
+ (Ashish Singhi via umamahesh)
+
+ HDFS-3336. hdfs launcher script will be better off not special casing
+ namenode command with regards to hadoop.security.logger (rvs via tucu)
+
+ HDFS-3330. If GetImageServlet throws an Error or RTE, response should not
+ have HTTP "OK" status. (todd)
+
+ HDFS-3351. NameNode#initializeGenericKeys should always set fs.defaultFS
+ regardless of whether HA or Federation is enabled. (atm)
+
+ HDFS-3332. NullPointerException in DN when directoryscanner is trying to
+ report bad blocks. (Amith D K via umamahesh)
+
+ HDFS-3359. DFSClient.close should close cached sockets. (todd)
+
+ HDFS-3350. In INode, add final to compareTo(..), equals(..) and hashCode(),
+ and remove synchronized from updatePermissionStatus(..). (szetszwo)
+
+ HDFS-3357. DataXceiver reads from client socket with incorrect/no timeout
+ (todd)
+
+ HDFS-3376. DFSClient fails to make connection to DN if there are many
+ unusable cached sockets (todd)
+
+ HDFS-3328. NPE in DataNode.getIpcPort. (eli)
+
+ HDFS-3396. FUSE build fails on Ubuntu 12.04. (Colin Patrick McCabe via eli)
+
+ HDFS-3395. NN doesn't start with HA+security enabled and HTTP address set to 0.0.0.0. (atm)
+
+ HDFS-3026. HA: Handle failure during HA state transition. (atm)
+
+ HDFS-860. fuse-dfs truncate behavior causes issues with scp.
+ (Brian Bockelman via eli)
+
+ BREAKDOWN OF HDFS-1623 SUBTASKS
+
+ HDFS-2179. Add fencing framework and mechanisms for NameNode HA. (todd)
+
+ HDFS-1974. Introduce active and standy states to the namenode. (suresh)
+
+ HDFS-2407. getServerDefaults and getStats don't check operation category (atm)
+
+ HDFS-1973. HA: HDFS clients must handle namenode failover and switch over to
+ the new active namenode. (atm)
+
+ HDFS-2301. Start/stop appropriate namenode services when transition to active
+ and standby states. (suresh)
+
+ HDFS-2231. Configuration changes for HA namenode. (suresh)
+
+ HDFS-2418. Change ConfiguredFailoverProxyProvider to take advantage of
+ HDFS-2231. (atm)
+
+ HDFS-2393. Mark appropriate methods of ClientProtocol with the idempotent
+ annotation. (atm)
+
+ HDFS-2523. Small NN fixes to include HAServiceProtocol and prevent NPE on
+ shutdown. (todd)
+
+ HDFS-2577. NN fails to start since it tries to start secret manager in
+ safemode. (todd)
+
+ HDFS-2582. Scope dfs.ha.namenodes config by nameservice (todd)
+
+ HDFS-2591. MiniDFSCluster support to mix and match federation with HA (todd)
+
+ HDFS-1975. Support for sharing the namenode state from active to standby.
+ (jitendra, atm, todd)
+
+ HDFS-1971. Send block report from datanode to both active and standby
+ namenodes. (sanjay, todd via suresh)
+
+ HDFS-2616. Change DatanodeProtocol#sendHeartbeat() to return HeartbeatResponse.
+ (suresh)
+
+ HDFS-2622. Fix TestDFSUpgrade in HA branch. (todd)
+
+ HDFS-2612. Handle refreshNameNodes in federated HA clusters (todd)
+
+ HDFS-2623. Add test case for hot standby capability (todd)
+
+ HDFS-2626. BPOfferService.verifyAndSetNamespaceInfo needs to be synchronized
+ (todd)
+
+ HDFS-2624. ConfiguredFailoverProxyProvider doesn't correctly stop
+ ProtocolTranslators (todd)
+
+ HDFS-2625. TestDfsOverAvroRpc failing after introduction of HeartbeatResponse
+ type (todd)
+
+ HDFS-2627. Determine DN's view of which NN is active based on heartbeat
+ responses (todd)
+
+ HDFS-2634. Standby needs to ingest latest edit logs before transitioning to
+ active (todd)
+
+ HDFS-2671. NN should throw StandbyException in response to RPCs in STANDBY
+ state (todd)
+
+ HDFS-2680. DFSClient should construct failover proxy with exponential backoff
+ (todd)
+
+ HDFS-2683. Authority-based lookup of proxy provider fails if path becomes
+ canonicalized (todd)
+
+ HDFS-2689. HA: BookKeeperEditLogInputStream doesn't implement isInProgress()
+ (atm)
+
+ HDFS-2602. NN should log newly-allocated blocks without losing BlockInfo (atm)
+
+ HDFS-2667. Fix transition from active to standby (todd)
+
+ HDFS-2684. Fix up some failing unit tests on HA branch (todd)
+
+ HDFS-2679. Add interface to query current state to HAServiceProtocol (eli via
+ todd)
+
+ HDFS-2677. Web UI should indicate the NN state. (eli via todd)
+
+ HDFS-2678. When a FailoverProxyProvider is used, DFSClient should not retry
+ connection ten times before failing over (atm via todd)
+
+ HDFS-2682. When a FailoverProxyProvider is used, Client should not retry for 45
+ times if it is timing out to connect to server. (Uma Maheswara Rao G via todd)
+
+ HDFS-2693. Fix synchronization issues around state transition (todd)
+
+ HDFS-1972. Fencing mechanism for block invalidations and replications (todd)
+
+ HDFS-2714. Fix test cases which use standalone FSNamesystems (todd)
+
+ HDFS-2692. Fix bugs related to failover from/into safe mode. (todd)
+
+ HDFS-2716. Configuration needs to allow different dfs.http.addresses for each
+ HA NN (todd)
+
+ HDFS-2720. Fix MiniDFSCluster HA support to work properly on Windows. (Uma
+ Maheswara Rao G via todd)
+
+ HDFS-2291. Allow the StandbyNode to make checkpoints in an HA setup. (todd)
+
+ HDFS-2709. Appropriately handle error conditions in EditLogTailer (atm via
+ todd)
+
+ HDFS-2730. Refactor shared HA-related test code into HATestUtil class (todd)
+
+ HDFS-2762. Fix TestCheckpoint timing out on HA branch. (Uma Maheswara Rao G via
+ todd)
+
+ HDFS-2724. NN web UI can throw NPE after startup, before standby state is
+ entered. (todd)
+
+ HDFS-2753. Fix standby getting stuck in safemode when blocks are written while
+ SBN is down. (Hari Mankude and todd via todd)
+
+ HDFS-2773. Reading edit logs from an earlier version should not leave blocks in
+ under-construction state. (todd)
+
+ HDFS-2775. Fix TestStandbyCheckpoints.testBothNodesInStandbyState failing
+ intermittently. (todd)
+
+ HDFS-2766. Test for case where standby partially reads log and then performs
+ checkpoint. (atm)
+
+ HDFS-2738. FSEditLog.selectinputStreams is reading through in-progress streams
+ even when non-in-progress are requested. (atm)
+
+ HDFS-2789. TestHAAdmin.testFailover is failing (eli)
+
+ HDFS-2747. Entering safe mode after starting SBN can NPE. (Uma Maheswara Rao G
+ via todd)
+
+ HDFS-2772. On transition to active, standby should not swallow ELIE. (atm)
+
+ HDFS-2767. ConfiguredFailoverProxyProvider should support NameNodeProtocol.
+ (Uma Maheswara Rao G via todd)
+
+ HDFS-2795. Standby NN takes a long time to recover from a dead DN starting up.
+ (todd)
+
+ HDFS-2592. Balancer support for HA namenodes. (Uma Maheswara Rao G via todd)
+
+ HDFS-2367. Enable the configuration of multiple HA cluster addresses. (atm)
+
+ HDFS-2812. When becoming active, the NN should treat all leases as freshly
+ renewed. (todd)
+
+ HDFS-2737. Automatically trigger log rolls periodically on the active NN. (todd
+ and atm)
+
+ HDFS-2820. Add a simple sanity check for HA config (todd)
+
+ HDFS-2688. Add tests for quota tracking in an HA cluster. (todd)
+
+ HDFS-2804. Should not mark blocks under-replicated when exiting safemode (todd)
+
+ HDFS-2807. Service level authorizartion for HAServiceProtocol. (jitendra)
+
+ HDFS-2809. Add test to verify that delegation tokens are honored after
+ failover. (jitendra and atm)
+
+ HDFS-2838. NPE in FSNamesystem when in safe mode. (Gregory Chanan via eli)
+
+ HDFS-2805. Add a test for a federated cluster with HA NNs. (Brandon Li via
+ jitendra)
+
+ HDFS-2841. HAAdmin does not work if security is enabled. (atm)
+
+ HDFS-2691. Fixes for pipeline recovery in an HA cluster: report RBW replicas
+ immediately upon pipeline creation. (todd)
+
+ HDFS-2824. Fix failover when prior NN died just after creating an edit log
+ segment. (atm via todd)
+
+ HDFS-2853. HA: NN fails to start if the shared edits dir is marked required
+ (atm via eli)
+
+ HDFS-2845. SBN should not allow browsing of the file system via web UI. (Bikas
+ Saha via atm)
+
+ HDFS-2742. HA: observed dataloss in replication stress test. (todd via eli)
+
+ HDFS-2870. Fix log level for block debug info in processMisReplicatedBlocks
+ (todd)
+
+ HDFS-2859. LOCAL_ADDRESS_MATCHER.match has NPE when called from
+ DFSUtil.getSuffixIDs when the host is incorrect (Bikas Saha via todd)
+
+ HDFS-2861. checkpointing should verify that the dfs.http.address has been
+ configured to a non-loopback for peer NN (todd)
+
+ HDFS-2860. TestDFSRollback#testRollback is failing. (atm)
+
+ HDFS-2769. HA: When HA is enabled with a shared edits dir, that dir should be
+ marked required. (atm via eli)
+
+ HDFS-2863. Failures observed if dfs.edits.dir and shared.edits.dir have same
+ directories. (Bikas Saha via atm)
+
+ HDFS-2874. Edit log should log to shared dirs before local dirs. (todd)
+
+ HDFS-2890. DFSUtil#getSuffixIDs should skip unset configurations. (atm)
+
+ HDFS-2792. Make fsck work. (atm)
+
+ HDFS-2808. HA: haadmin should use namenode ids. (eli)
+
+ HDFS-2819. Document new HA-related configs in hdfs-default.xml. (eli)
+
+ HDFS-2752. HA: exit if multiple shared dirs are configured. (eli)
+
+ HDFS-2894. HA: automatically determine the nameservice Id if only one
+ nameservice is configured. (eli)
+
+ HDFS-2733. Document HA configuration and CLI. (atm)
+
+ HDFS-2794. Active NN may purge edit log files before standby NN has a chance to
+ read them (todd)
+
+ HDFS-2901. Improvements for SBN web UI - not show under-replicated/missing
+ blocks. (Brandon Li via jitendra)
+
+ HDFS-2905. HA: Standby NN NPE when shared edits dir is deleted. (Bikas Saha via
+ jitendra)
+
+ HDFS-2579. Starting delegation token manager during safemode fails. (todd)
+
+ HDFS-2510. Add HA-related metrics. (atm)
+
+ HDFS-2924. Standby checkpointing fails to authenticate in secure cluster.
+ (todd)
+
+ HDFS-2915. HA: TestFailureOfSharedDir.testFailureOfSharedDir() has race
+ condition. (Bikas Saha via jitendra)
+
+ HDFS-2912. Namenode not shutting down when shared edits dir is inaccessible.
+ (Bikas Saha via atm)
+
+ HDFS-2917. HA: haadmin should not work if run by regular user (eli)
+
+ HDFS-2939. TestHAStateTransitions fails on Windows. (Uma Maheswara Rao G via
+ atm)
+
+ HDFS-2947. On startup NN throws an NPE in the metrics system. (atm)
+
+ HDFS-2942. TestActiveStandbyElectorRealZK fails if build dir does not exist.
+ (atm)
+
+ HDFS-2948. NN throws NPE during shutdown if it fails to startup (todd)
+
+ HDFS-2909. HA: Inaccessible shared edits dir not getting removed from FSImage
+ storage dirs upon error. (Bikas Saha via jitendra)
+
+ HDFS-2934. Allow configs to be scoped to all NNs in the nameservice. (todd)
+
+ HDFS-2935. Shared edits dir property should be suffixed with nameservice and
+ namenodeID (todd)
+
+ HDFS-2928. ConfiguredFailoverProxyProvider should not create a NameNode proxy
+ with an underlying retry proxy. (Uma Maheswara Rao G via atm)
+
+ HDFS-2955. IllegalStateException during standby startup in getCurSegmentTxId.
+ (Hari Mankude via atm)
+
+ HDFS-2937. TestDFSHAAdmin needs tests with MiniDFSCluster. (Brandon Li via
+ suresh)
+
+ HDFS-2586. Add protobuf service and implementation for HAServiceProtocol.
+ (suresh via atm)
+
+ HDFS-2952. NN should not start with upgrade option or with a pending an
+ unfinalized upgrade. (atm)
+
+ HDFS-2974. MiniDFSCluster does not delete standby NN name dirs during format.
+ (atm)
+
+ HDFS-2929. Stress test and fixes for block synchronization (todd)
+
+ HDFS-2972. Small optimization building incremental block report (todd)
+
+ HDFS-2973. Re-enable NO_ACK optimization for block deletion. (todd)
+
+ HDFS-2922. HA: close out operation categories (eli)
+
+ HDFS-2993. HA: BackupNode#checkOperation should permit CHECKPOINT operations
+ (eli)
+
+ HDFS-2904. Client support for getting delegation tokens. (todd)
+
+ HDFS-3013. HA: NameNode format doesn't pick up
+ dfs.namenode.name.dir.NameServiceId configuration (Mingjie Lai via todd)
+
+ HDFS-3019. Fix silent failure of TestEditLogJournalFailures (todd)
+
+ HDFS-2958. Sweep for remaining proxy construction which doesn't go through
+ failover path. (atm)
+
+ HDFS-2920. fix remaining TODO items. (atm and todd)
+
+ HDFS-3027. Implement a simple NN health check. (atm)
+
+ HDFS-3023. Optimize entries in edits log for persistBlocks call. (todd)
+
+ HDFS-2979. Balancer should use logical uri for creating failover proxy with HA
+ enabled. (atm)
+
+ HDFS-3035. Fix failure of TestFileAppendRestart due to OP_UPDATE_BLOCKS (todd)
+
+ HDFS-3039. Address findbugs and javadoc warnings on branch. (todd via atm)
+
+Release 0.23.10 - UNRELEASED
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ HDFS-5010. Reduce the frequency of getCurrentUser() calls from namenode
+ (kihwal)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-4998. TestUnderReplicatedBlocks fails intermittently (kihwal)
+
+Release 0.23.9 - UNRELEASED
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-4867. metaSave NPEs when there are invalid blocks in repl queue.
+ (Plamen Jeliazkov and Ravi Prakash via shv)
+
+ HDFS-4862. SafeModeInfo.isManual() returns true when resources are low even
+ if it wasn't entered into manually (Ravi Prakash via kihwal)
+
+ HDFS-4832. Namenode doesn't change the number of missing blocks in
+ safemode when DNs rejoin or leave (Ravi Prakash via kihwal)
+
+ HDFS-4878. On Remove Block, block is not removed from neededReplications
+ queue. (Tao Luo via shv)
+
+ HDFS-4205. fsck fails with symlinks. (jlowe)
+
+Release 0.23.8 - 2013-06-05
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ HDFS-4714. Log short messages in Namenode RPC server for exceptions
+ meant for clients. (kihwal)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-4477. Secondary namenode may retain old tokens (daryn via kihwal)
+
+ HDFS-4699. TestPipelinesFailover#testPipelineRecoveryStress fails
+ sporadically (Chris Nauroth via kihwal)
+
+ HDFS-4805. Webhdfs client is fragile to token renewal errors
+ (daryn via kihwal)
+
+ HDFS-3875. Issue handling checksum errors in write pipeline. (kihwal)
+
+ HDFS-4807. createSocketForPipeline() should not include timeout extension
+ on connect. (Cristina L. Abad via kihwal)
+
+Release 0.23.7 - 2013-04-18
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ HDFS-4532. RPC call queue may fill due to current user lookup (daryn)
+
+ BUG FIXES
+
+ HDFS-4288. NN accepts incremental BR as IBR in safemode (daryn via kihwal)
+
+ HDFS-4495. Allow client-side lease renewal to be retried beyond soft-limit
+ (kihwal)
+
+ HDFS-4128. 2NN gets stuck in inconsistent state if edit log replay fails
+ in the middle (kihwal via daryn)
+
+ HDFS-4542. Webhdfs doesn't support secure proxy users (Daryn Sharp via
+ kihwal)
+
+ HDFS-4560. Webhdfs cannot use tokens obtained by another user (daryn)
+
+ HDFS-4566. Webdhfs token cancelation should use authentication (daryn)
+
+ HDFS-4567. Webhdfs does not need a token for token operations (daryn via
+ kihwal)
+
+ HDFS-4577. Webhdfs operations should declare if authentication is required
+ (daryn via kihwal)
+
+ HDFS-3344. Unreliable corrupt blocks counting in TestProcessCorruptBlocks
+ (kihwal)
+
+ HDFS-3367. WebHDFS doesn't use the logged in user when opening
+ connections (daryn)
+
+ HDFS-4581. checkDiskError should not be called on network errors (Rohit
+ Kochar via kihwal)
+
+ HDFS-4649. Webhdfs cannot list large directories (daryn via kihwal)
+
+ HDFS-4548. Webhdfs doesn't renegotiate SPNEGO token (daryn via kihwal)
+
+Release 0.23.6 - 2013-02-06
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-4247. saveNamespace should be tolerant of dangling lease (daryn)
+
+ HDFS-4248. Renaming directories may incorrectly remove the paths in leases
+ under the tree. (daryn via szetszwo)
+
+ HDFS-4385. Maven RAT plugin is not checking all source files (tgraves)
+
+ HDFS-4426. Secondary namenode shuts down immediately after startup.
+ (Arpit Agarwal via suresh)
+
+Release 0.23.5 - 2012-11-28
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-4080. Add a separate logger for block state change logs to enable turning
+ off those logs. (Kihwal Lee via suresh)
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ HDFS-4075. Reduce recommissioning overhead (Kihwal Lee via daryn)
+
+ HDFS-3990. NN's health report has severe performance problems (daryn)
+
+ HDFS-4181. LeaseManager tries to double remove and prints extra messages
+ (Kihwal Lee via daryn)
+
+ BUG FIXES
+
+ HDFS-3829. TestHftpURLTimeouts fails intermittently with JDK7 (Trevor
+ Robinson via tgraves)
+
+ HDFS-3824. TestHftpDelegationToken fails intermittently with JDK7 (Trevor
+ Robinson via tgraves)
+
+ HDFS-3224. Bug in check for DN re-registration with different storage ID
+ (jlowe)
+
+ HDFS-4090. getFileChecksum() result incompatible when called against
+ zero-byte files. (Kihwal Lee via daryn)
+
+ HDFS-4172. namenode does not URI-encode parameters when building URI for
+ datanode request (Derek Dagit via bobby)
+
+ HDFS-4182. SecondaryNameNode leaks NameCache entries (bobby)
+
+ HDFS-4186. logSync() is called with the write lock held while releasing
+ lease (Kihwal Lee via daryn)
+
+Release 0.23.4
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-3831. Failure to renew tokens due to test-sources left in classpath
+ (jlowe via bobby)
+
+Release 0.23.3
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ HDFS-2505. Add a test to verify getFileChecksum(..) with ViewFS. (Ravi
+ Prakash via szetszwo)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-3166. Add timeout to Hftp connections. (Daryn Sharp via szetszwo)
+
+ HDFS-3176. Use MD5MD5CRC32FileChecksum.readFields() in JsonUtil . (Kihwal
+ Lee via szetszwo)
+
+ HDFS-2652. Add support for host-based delegation tokens. (Daryn Sharp via
+ szetszwo)
+
+ HDFS-3308. Uses canonical URI to select delegation tokens in HftpFileSystem
+ and WebHdfsFileSystem. (Daryn Sharp via szetszwo)
+
+ HDFS-3312. In HftpFileSystem, the namenode URI is non-secure but the
+ delegation tokens have to use secure URI. (Daryn Sharp via szetszwo)
+
+ HDFS-3318. Use BoundedInputStream in ByteRangeInputStream, otherwise, it
+ hangs on transfers >2 GB. (Daryn Sharp via szetszwo)
+
+ HDFS-3321. Fix safe mode turn off tip message. (Ravi Prakash via szetszwo)
+
+ HDFS-3334. Fix ByteRangeInputStream stream leakage. (Daryn Sharp via
+ szetszwo)
+
+ HDFS-3331. In namenode, check superuser privilege for setBalancerBandwidth
+ and acquire the write lock for finalizeUpgrade. (szetszwo)
+
+ HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is
+ racy. (atm)
+
+ HDFS-3577. In DatanodeWebHdfsMethods, use MessageBodyWriter instead of
+ StreamingOutput, otherwise, it will fail to transfer large files.
+ (szetszwo)
+
+ HDFS-3646. LeaseRenewer can hold reference to inactive DFSClient
+ instances forever. (Kihwal Lee via daryn)
+
+ HDFS-3696. Set chunked streaming mode in WebHdfsFileSystem write operations
+ to get around a Java library bug causing OutOfMemoryError. (szetszwo)
+
+ HDFS-3553. Hftp proxy tokens are broken (daryn)
+
+ HDFS-3718. Datanode won't shutdown because of runaway DataBlockScanner
+ thread (Kihwal Lee via daryn)
+
+ HDFS-3861. Deadlock in DFSClient (Kihwal Lee via daryn)
+
+ HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn)
+
+ HDFS-3852. TestHftpDelegationToken is broken after HADOOP-8225 (daryn)
+
+ HDFS-3890. filecontext mkdirs doesn't apply umask as expected
+ (Tom Graves via daryn)
+
+Release 0.23.2 - UNRELEASED
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-2887. FSVolume, is a part of FSDatasetInterface implementation, should
+ not be referred outside FSDataset. A new FSVolumeInterface is defined.
+ The BlockVolumeChoosingPolicy.chooseVolume(..) method signature is also
+ updated. (szetszwo)
+
+ NEW FEATURES
+
+ HDFS-2943. Expose last checkpoint time and transaction stats as JMX
+ metrics. (atm)
+
+ IMPROVEMENTS
+
+ HDFS-2931. Switch DataNode's BlockVolumeChoosingPolicy to private-audience.
+ (harsh via szetszwo)
+
+ HDFS-2655. BlockReaderLocal#skip performs unnecessary IO. (Brandon Li
+ via jitendra)
+
+ HDFS-2725. hdfs script usage information is missing the information
+ about "dfs" command (Prashant Sharma via stevel)
+
+ HDFS-2907. Add a conf property dfs.datanode.fsdataset.factory to make
+ FSDataset in Datanode pluggable. (szetszwo)
+
+ HDFS-2985. Improve logging when replicas are marked as corrupt. (todd)
+
+ HDFS-3098. Update and add tests for HADOOP-8173. (Daryn Sharp via szetszwo)
+
+ HDFS-3104. Add tests for HADOOP-8175. (Daryn Sharp via szetszwo)
+
+ HDFS-3066. Cap space usage of default log4j rolling policy.
+ (Patrick Hunt via eli)
+
+ OPTIMIZATIONS
+
+ HDFS-3024. Improve performance of stringification in addStoredBlock (todd)
+
+ BUG FIXES
+ HDFS-2923. Namenode IPC handler count uses the wrong configuration key
+ (todd)
+
+ HDFS-2764. TestBackupNode is racy. (atm)
+
+ HDFS-2869. Fix an error in the webhdfs docs for the mkdir op (harsh)
+
+ HDFS-776. Fix exception handling in Balancer. (Uma Maheswara Rao G
+ via szetszwo)
+
+ HDFS-2815. Namenode sometimes oes not come out of safemode during
+ NN crash + restart. (Uma Maheswara Rao via suresh)
+
+ HDFS-2950. Secondary NN HTTPS address should be listed as a
+ NAMESERVICE_SPECIFIC_KEY. (todd)
+
+ HDFS-2525. Race between BlockPoolSliceScanner and append. (Brandon Li
+ via jitendra)
+
+ HDFS-2938. Recursive delete of a large directory make namenode
+ unresponsive. (Hari Mankude via suresh)
+
+ HDFS-2969. ExtendedBlock.equals is incorrectly implemented (todd)
+
+ HDFS-2944. Typo in hdfs-default.xml causes
+ dfs.client.block.write.replace-datanode-on-failure.enable to be mistakenly
+ disabled. (atm)
+
+ HDFS-2981. In hdfs-default.xml, the default value of
+ dfs.client.block.write.replace-datanode-on-failure.enable should be true.
+ (szetszwo)
+
+ HDFS-3008. Negative caching of local addrs doesn't work. (eli)
+
+ HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type
+ to application/octet-stream instead of application/json. (szetszwo)
+
+ HDFS-2991. Fix case where OP_ADD would not be logged in append(). (todd)
+
+ HDFS-3012. Exception while renewing delegation token. (Bobby Evans via
+ jitendra)
+
+ HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the
+ lease soft-limit. (Kihwal Lee via szetszwo)
+
+ HDFS-2038. Update TestHDFSCLI to handle relative paths with globs.
+ (Kihwal Lee via szetszwo)
+
+ HDFS-3101. Cannot read empty file using WebHDFS. (szetszwo)
+
+ HDFS-3160. httpfs should exec catalina instead of forking it.
+ (Roman Shaposhnik via eli)
+
+Release 0.23.1 - 2012-02-17
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ HDFS-2316. [umbrella] WebHDFS: a complete FileSystem implementation for
+ accessing HDFS over HTTP (szetszwo)
+
+ HDFS-2594. Support getDelegationTokens and createSymlink in WebHDFS.
+ (szetszwo)
+
+ HDFS-2545. Change WebHDFS to support multiple namenodes in federation.
+ (szetszwo)
+
+ HDFS-2178. Contributing Hoop to HDFS, replacement for HDFS proxy
+ with read/write capabilities. (tucu)
+
+
+ IMPROVEMENTS
+ HDFS-2560. Refactor BPOfferService to be a static inner class (todd)
+
+ HDFS-2544. Hadoop scripts unconditionally source
+ "$bin"/../libexec/hadoop-config.sh. (Bruno Mahé via tomwhite)
+
+ HDFS-2543. HADOOP_PREFIX cannot be overridden. (Bruno Mahé via tomwhite)
+
+ HDFS-2562. Refactor DN configuration variables out of DataNode class
+ (todd)
+
+ HDFS-2563. Some cleanup in BPOfferService. (todd)
+
+ HDFS-2568. Use a set to manage child sockets in XceiverServer.
+ (harsh via eli)
+
+ HDFS-2454. Move maxXceiverCount check to before starting the
+ thread in dataXceiver. (harsh via eli)
+
+ HDFS-2570. Add descriptions for dfs.*.https.address in hdfs-default.xml.
+ (eli)
+
+ HDFS-2536. Remove unused imports. (harsh via eli)
+
+ HDFS-2566. Move BPOfferService to be a non-inner class. (todd)
+
+ HDFS-2552. Add Forrest doc for WebHDFS REST API. (szetszwo)
+
+ HDFS-2587. Add apt doc for WebHDFS REST API. (szetszwo)
+
+ HDFS-2604. Add a log message to show if WebHDFS is enabled and a
+ configuration section in the forrest doc. (szetszwo)
+
+ HDFS-2511. Add dev script to generate HDFS protobufs. (tucu)
+
+ HDFS-2654. Make BlockReaderLocal not extend RemoteBlockReader2. (eli)
+
+ HDFS-2675. Reduce warning verbosity when double-closing edit logs
+ (todd)
+
+ HDFS-2335. DataNodeCluster and NNStorage always pull fresh entropy.
+ (Uma Maheswara Rao G via eli)
+
+ HDFS-2574. Remove references to some deprecated properties in conf
+ templates and defaults files. (Joe Crobak via harsh)
+
+ HDFS-2722. HttpFs should not be using an int for block size. (harsh)
+
+ HDFS-2710. Add HDFS tests related to HADOOP-7933. (Siddarth Seth via
+ suresh)
+
+ HDFS-2349. Corruption detected during block transfers between DNs
+ should log a WARN instead of INFO. (harsh)
+
+ HDFS-2729. Update BlockManager's comments regarding the invalid block
+ set (harsh)
+
+ HDFS-2726. Fix a logging issue under DFSClient's createBlockOutputStream
+ method (harsh)
+
+ HDFS-554. Use System.arraycopy in BlockInfo.ensureCapacity. (harsh)
+
+ HDFS-1314. Make dfs.blocksize accept size-indicating prefixes.
+ (Sho Shimauchi via harsh)
+
+ HDFS-69. Improve the 'dfsadmin' commandline help. (harsh)
+
+ HDFS-2788. HdfsServerConstants#DN_KEEPALIVE_TIMEOUT is dead code. (eli)
+
+ HDFS-362. FSEditLog should not writes long and short as UTF8, and should
+ not use ArrayWritable for writing non-array items. (Uma Maheswara Rao G
+ via szetszwo)
+
+ HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging.
+ (Jimmy Xiang via todd)
+
+ HDFS-2817. Combine the two TestSafeMode test suites. (todd)
+
+ HDFS-2818. Fix a missing space issue in HDFS webapps' title tags. (Devaraj K via harsh)
+
+ HDFS-2397. Undeprecate SecondaryNameNode. (eli)
+
+ HDFS-2814. NamenodeMXBean does not account for svn revision in the version
+ information. (Hitesh Shah via jitendra)
+
+ HDFS-2784. Update hftp and hdfs for host-based token support.
+ (Kihwal Lee via jitendra)
+
+ HDFS-2785. Update webhdfs and httpfs for host-based token support.
+ (Robert Joseph Evans via jitendra)
+
+ HDFS-2868. Expose xceiver counts via the DataNode MXBean. (harsh)
+
+ HDFS-2786. Fix host-based token incompatibilities in DFSUtil. (Kihwal
+ Lee via jitendra)
+
+ HDFS-208. name node should warn if only one dir is listed in dfs.name.dir.
+ (Uma Maheswara Rao G via eli)
+
+ HDFS-3139. Minor Datanode logging improvement. (eli)
+
+ OPTIMIZATIONS
+
+ HDFS-2130. Switch default checksum to CRC32C. (todd)
+
+ HDFS-2533. Remove needless synchronization on some FSDataSet methods.
+ (todd)
+
+ HDFS-2129. Simplify BlockReader to not inherit from FSInputChecker.
+ (todd)
+
+ HDFS-2246. Enable reading a block directly from local file system
+ for a client on the same node as the block file. (Andrew Purtell,
+ Suresh Srinivas and Jitendra Nath Pandey via szetszwo)
+
+ HDFS-2825. Add test hook to turn off the writer preferring its local
+ DN. (todd)
+
+ HDFS-2826. Add test case for HDFS-1476 (safemode can initialize
+ replication queues before exiting) (todd)
+
+ HDFS-2864. Remove some redundant methods and the constant METADATA_VERSION
+ from FSDataset. (szetszwo)
+
+ HDFS-2879. Change FSDataset to package private. (szetszwo)
+
+ BUG FIXES
+
+ HDFS-2541. For a sufficiently large value of blocks, the DN Scanner
+ may request a random number with a negative seed value. (harsh via eli)
+
+ HDFS-2502. hdfs-default.xml should include dfs.name.dir.restore.
+ (harsh via eli)
+
+ HDFS-2567. When 0 DNs are available, show a proper error when
+ trying to browse DFS via web UI. (harsh via eli)
+
+ HDFS-2575. DFSTestUtil may create empty files (todd)
+
+ HDFS-2588. hdfs jsp pages missing DOCTYPE. (Dave Vronay via mattf)
+
+ HDFS-2590. Fix the missing links in the WebHDFS forrest doc. (szetszwo)
+
+ HDFS-2596. TestDirectoryScanner doesn't test parallel scans. (eli)
+
+ HDFS-2606. webhdfs client filesystem impl must set the content-type
+ header for create/append. (tucu)
+
+ HDFS-2614. hadoop dist tarball is missing hdfs headers. (tucu)
+
+ HDFS-2653. DFSClient should cache whether addrs are non-local when
+ short-circuiting is enabled. (eli)
+
+ HDFS-2649. eclipse:eclipse build fails for hadoop-hdfs-httpfs.
+ (Jason Lowe via eli)
+
+ HDFS-2640. Javadoc generation hangs. (tomwhite)
+
+ HDFS-2553. Fix BlockPoolSliceScanner spinning in a tight loop (Uma
+ Maheswara Rao G via todd)
+
+ HDFS-2658. HttpFS introduced 70 javadoc warnings. (tucu)
+
+ HDFS-2706. Use configuration for blockInvalidateLimit if it is set.
+ (szetszwo)
+
+ HDFS-2646. Hadoop HttpFS introduced 4 findbug warnings. (tucu)
+
+ HDFS-2657. TestHttpFSServer and TestServerWebApp are failing on trunk. (tucu)
+
+ HDFS-2705. HttpFS server should check that upload requests have correct
+ content-type. (tucu)
+
+ HDFS-2707. HttpFS should read the hadoop-auth secret from a file instead
+ inline from the configuration. (tucu)
+
+ HDFS-2790. FSNamesystem.setTimes throws exception with wrong
+ configuration name in the message. (Arpit Gupta via eli)
+
+ HDFS-2810. Leases not getting renewed properly by clients (todd)
+
+ HDFS-2751. Datanode may incorrectly drop OS cache behind reads
+ even for short reads. (todd)
+
+ HDFS-2816. Fix missing license header in httpfs findbugsExcludeFile.xml.
+ (hitesh via tucu)
+
+ HDFS-2822. processMisReplicatedBlock incorrectly identifies
+ under-construction blocks as under-replicated. (todd)
+
+ HDFS-442. dfsthroughput in test jar throws NPE (harsh)
+
+ HDFS-2836. HttpFSServer still has 2 javadoc warnings in trunk.
+ (revans2 via tucu)
+
+ HDFS-2837. mvn javadoc:javadoc not seeing LimitedPrivate class.
+ (revans2 via tucu)
+
+ HDFS-2840. TestHostnameFilter should work with localhost or
+ localhost.localdomain (tucu)
+
+ HDFS-2791. If block report races with closing of file, replica is
+ incorrectly marked corrupt. (todd)
+
+ HDFS-2827. When the parent of a directory is the root, renaming the
+ directory results in leases updated incorrectly. (Uma Maheswara Rao G
+ via szetszwo)
+
+ HDFS-2835. Fix findbugs and javadoc issue with GetConf.java.
+ (suresh)
+
+ HDFS-2889. getNumCurrentReplicas is package private but should be public on
+ 0.23 (see HDFS-2408). (Gregory Chanan via atm)
+
+ HDFS-2893. The start/stop scripts don't start/stop the 2NN when
+ using the default configuration. (eli)
+
+Release 0.23.0 - 2011-11-01
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-1526. Dfs client name for a map/reduce task should be unique
+ among threads. (hairong)
+
+ HDFS-1536. Improve HDFS WebUI. (hairong)
+
+ HDFS-2210. Remove hdfsproxy. (eli)
+
+ HDFS-1073. Redesign the NameNode's storage layout for image checkpoints
+ and edit logs to introduce transaction IDs and be more robust.
+ Please see HDFS-1073 section below for breakout of individual patches.
+
+ NEW FEATURES
+
+ HDFS-1359. Add BlockPoolID to Block. (suresh)
+
+ HDFS-1365. Federation: propose ClusterID and BlockPoolID format
+ (Tanping via boryas)
+
+ HDFS-1394. Federation: modify -format option for namenode to generated
+ new blockpool id and accept newcluster (boryas)
+
+ HDFS-1400. Federation: DataTransferProtocol uses ExtendedBlockPool to
+ include BlockPoolID in the protocol. (suresh)
+
+ HDFS-1428. Federation : add cluster ID and block pool ID into
+ Name node web UI(Tanping via boryas)
+
+ HDFS-1450. Federation: Introduce block pool ID into FSDatasetInterface.
+ (suresh)
+
+ HDFS-1632. Federation: data node storage structure changes and
+ introduce block pool storage. (Tanping via suresh)
+
+ HDFS-1634. Federation: Convert single threaded DataNode into
+ per BlockPool thread model.(boryas)
+
+ HDFS-1637. Federation: FSDataset in Datanode should be created after
+ initial handshake with namenode. (boryas and jitendra)
+
+ HDFS-1653. Federation: Block received message from datanode sends invalid
+ DatanodeRegistration. (Tanping via suresh)
+
+ HDFS-1645. Federation: DatanodeCommond.Finalize needs to include
+ BlockPoolId. (suresh)
+
+ HDFS-1638. Federation: DataNode.handleDiskError needs to inform
+ ALL namenodes if a disk failed (boryas)
+
+ HDFS-1647. Federation: Multiple namenode configuration. (jitendra)
+
+ HDFS-1639. Federation: Add block pool management to FSDataset. (suresh)
+
+ HDFS-1648. Federation: Only DataStorage must be locked using in_use.lock
+ and no locks must be associated with BlockPoolStorage. (Tanping via suresh)
+
+ HDFS-1641. Federation: Datanode fields that are no longer used should
+ be removed (boryas)
+
+ HDFS-1642. Federation: add Datanode.getDNRegistration(String bpid)
+ method (boryas)
+
+ HDFS-1643. Federation: remove namenode argument from DataNode
+ constructor (boryas)
+
+ HDFS-1657. Federation: Tests that corrupt block files fail due to changed
+ file path in federation. (suresh)
+
+ HDFS-1661. Federation: Remove unnecessary TODO:FEDERATION comments.
+ (jitendra)
+
+ HDFS-1660. Federation: Datanode doesn't start with two namenodes (boryas)
+
+ HDFS-1650. Federation: TestReplication fails. (Tanping via suresh)
+
+ HDFS-1651. Federation: Tests fail due to null pointer exception in
+ Datnode#shutdown() method. (Tanping via suresh)
+
+ HDFS-1649. Federation: Datanode command to refresh namenode list at
+ the datanode. (jitendra)
+
+ HDFS-1646. Federation: MiniDFSClsuter#waitActive() waits for ever
+ with the introduction of BPOfferService in datanode. (suresh)
+
+ HDFS-1659. Federation: BPOfferService exits after one iteration
+ incorrectly. (Tanping via suresh)
+
+ HDFS-1654. Federation: Fix TestDFSUpgrade and TestDFSRollback failures.
+ (suresh)
+
+ HDFS-1668. Federation: Datanodes sends block pool usage information
+ to the namenode in heartbeat. (suresh)
+
+ HDFS-1669. Federation: Fix TestHftpFileSystem failure. (suresh)
+
+ HDFS-1670. Federation: remove dnRegistration from Datanode (boryas)
+
+ HDFS-1662. Federation: fix unit test case, TestCheckpoint
+ and TestDataNodeMXBean (tanping via boryas)
+
+ HDFS-1671. Federation: shutdown in DataNode should be able to
+ shutdown individual BP threads as well as the whole DN (boryas).
+
+ HDFS-1663. Federation: Rename getPoolId() everywhere to
+ getBlockPoolId() (tanping via boryas)
+
+ HDFS-1652. FederationL Add support for multiple namenodes in
+ MiniDFSCluster. (suresh)
+
+ HDFS-1672. Federation: refactor stopDatanode(name) to work
+ with multiple Block Pools (boryas)
+
+ HDFS-1687. Federation: DirectoryScanner changes for
+ federation (Matt Foley via boryas)
+
+ HDFS-1626. Make BLOCK_INVALIDATE_LIMIT configurable. (szetszwo)
+
+ HDFS-1655. Federation: DatablockScanner should scan blocks for
+ all the block pools. (jitendra)
+
+ HDFS-1664. Federation: Add block pool storage usage to Namenode WebUI.
+ (Tanping via suresh)
+
+ HDFS-1674. Federation: Rename BlockPool class to BlockPoolSlice.
+ (jghoman, Tanping via suresh)
+
+ HDFS-1673. Federation: Datanode changes to track block token secret per
+ namenode. (suresh)
+
+ HDFS-1677. Federation: Fix TestFsck and TestListCorruptFileBlocks
+ failures. (Tanping via suresh)
+
+ HDFS-1678. Federation: Remove unnecessary #getBlockpool()
+ for NameNodeMXBean in FSNameSystem. (Tanping via Suresh)
+
+ HDFS-1688. Federation: Fix failures in fault injection tests,
+ TestDiskError, TestDatanodeRestart and TestDFSTartupVersions. (suresh)
+
+ HDFS-1696. Federation: when build version doesn't match -
+ datanode should wait (keep connecting) untill NN comes up
+ with the right version (boryas)
+
+ HDFS-1681. Balancer: support per pool and per node policies. (szetszwo)
+
+ HDFS-1695. Federation: Fix testOIV and TestDatanodeUtils
+ (jhoman and tanping via boryas)
+
+ HDFS-1699. Federation: Fix failure of TestBlockReport.
+ (Matt Foley via suresh)
+
+ HDFS-1698. Federation: TestOverReplicatedBlocks and TestWriteToReplica
+ failing. (jhoman and jitendra)
+
+ HDFS-1701. Federation: Fix TestHeartbeathandling.
+ (Erik Steffl and Tanping Wang via suresh)
+
+ HDFS-1693. Federation: Fix TestDFSStorageStateRecovery failure. (suresh)
+
+ HDFS-1694. Federation: SimulatedFSDataset changes to work with
+ federation and multiple block pools. (suresh)
+
+ HDFS-1689. Federation: Configuration for namenodes. (suresh and jitendra)
+
+ HDFS-1682. Change Balancer CLI for multiple namenodes and balancing
+ policy. (szetszwo)
+
+ HDFS-1697. Federation: fix TestBlockRecovery (boryas)
+
+ HDFS-1702. Federation: fix TestBackupNode and TestRefreshNamendoes
+ failures. (suresh)
+
+ HDFS-1706. Federation: TestFileAppend2, TestFileAppend3 and
+ TestBlockTokenWithDFS failing. (jitendra)
+
+ HDFS-1704. Federation: Add a tool that lists namenodes, secondary and
+ backup from configuration file. (suresh)
+
+ HDFS-1711. Federation: create method for updating machine name in
+ DataNode.java (boryas)
+
+ HDFS-1712. Federation: when looking up datanode we should use machineNmae
+ (in testOverReplicatedBlocks) (boryas)
+
+ HDFS-1709. Federation: Error "nnaddr url param is null" when clicking on a
+ node from NN Live Node Link. (jitendra)
+
+ HDFS-1714. Federation: refactor upgrade object in DataNode (boryas)
+
+ HDFS-1715. Federation: warning/error not generated when datanode sees
+ inconsistent/different Cluster ID between namenodes (boryas)
+
+ HDFS-1715. Federation: warning/error not generated when datanode sees
+ inconsistent/different Cluster ID between namenodes (boryas)
+
+ HDFS-1716. Federation: Add decommission tests for federated namenodes.
+ (suresh)
+
+ HDFS-1713. Federation: Prevent DataBlockScanner from running in tight loop.
+ (jitendra)
+
+ HDFS-1721. Federation: Configuration for principal names should not be
+ namenode specific. (jitendra)
+
+ HDFS-1717. Federation: FSDataset volumeMap access is not synchronized
+ correctly. (suresh)
+
+ HDFS-1722. Federation: Add flag to MiniDFSCluser to differentiate between
+ federation and non-federation modes. (boryas via suresh)
+
+ HDFS-1718. Federation: MiniDFSCluster#waitActive() bug causes some tests
+ to fail. (suresh)
+
+ HDFS-1719. Federation: Fix TestDFSRemove that fails intermittently.
+ (suresh)
+
+ HDFS-1720. Federation: FSVolumeSet volumes is not synchronized correctly.
+ (suresh)
+
+ HDFS-1700. Federation: fsck needs to work with federation changes.
+ (Matt Foley via suresh)
+
+ HDFS-1482. Add listCorruptFileBlocks to DistributedFileSystem.
+ (Patrick Kling via hairong)
+
+ HDFS-1448. Add a new tool Offline Edits Viewer (oev). (Erik Steffl
+ via szetszwo)
+
+ HDFS-1735. Federation: merge FSImage change in federation to
+ FSImage+NNStorage refactoring in trunk. (suresh)
+
+ HDFS-1737. Federation: Update the layout version for federation
+ changes. (suresh)
+
+ HDFS-1744. Federation: Add new layout version to offline image viewer
+ and edits viewer. (suresh)
+
+ HDFS-1745. Federation: Fix fault injection test failures. (suresh)
+
+ HDFS-1746. Federation: TestFileAppend3 fails intermittently. (jitendra)
+
+ HDFS-1703. Improve start/stop scripts and add decommission tool for
+ federation. (Tanping Wang, Erik Steffl via suresh)
+
+ HDFS-1749. Federation: TestListCorruptFileBlocks failing in federation
+ branch. (jitendra)
+
+ HDFS-1754. Federation: testFsck fails. (boryas)
+
+ HDFS-1755. Federation: The BPOfferService must always connect to namenode
+ as the login user. (jitendra)
+
+ HDFS-1675. Support transferring RBW between datanodes. (szetszwo)
+
+ HDFS-1791. Federation: Add command to delete block pool directories
+ from a datanode. (jitendra)
+
+ HDFS-1761. Add a new DataTransferProtocol operation, Op.TRANSFER_BLOCK,
+ for transferring RBW/Finalized with acknowledgement and without using RPC.
+ (szetszwo)
+
+ HDFS-1813. Federation: Authentication using BlockToken in RPC to datanode
+ fails. (jitendra)
+
+ HDFS-1630. Support fsedits checksum. (hairong)
+
+ HDFS-1606. Provide a stronger data guarantee in the write pipeline by
+ adding a new datanode when an existing datanode failed. (szetszwo)
+
+ HDFS-1442. Api to get delegation token in Hdfs class. (jitendra)
+
+ HDFS-1070. Speedup namenode image loading and saving by storing only
+ local file names. (hairong)
+
+ HDFS-1751. Intrinsic limits for HDFS files, directories (daryn via boryas).
+
+ HDFS-1873. Federation: Add cluster management web console.
+ (Tanping Wang via suresh)
+
+ HDFS-1911 HDFS tests for the newly added viewfs
+
+ HDFS-1814. Add "hdfs groups" command to query the server-side groups
+ resolved for a user. (Aaron T. Myers via todd)
+
+ HDFS-1914. Federation: namenode storage directories must be configurable
+ specific to name service. (suresh)
+
+ HDFS-1963. Create RPM and Debian packages for HDFS. Changes deployment
+ layout to be consistent across the binary tgz, rpm, and deb.
+ (Eric Yang via omalley)
+
+ HDFS-2058. Change Data Transfer wire protocol to use protocol buffers.
+ (todd)
+
+ HDFS-2055. Add hflush support to libhdfs. (Travis Crawford via eli)
+
+ HDFS-2083. Query JMX statistics over http via JMXJsonServlet. (tanping)
+
+ HDFS-2156. Make hdfs and mapreduce rpm only depend on the same major
+ version for common and hdfs. (eyang via omalley)
+
+ HDFS-2202. Add a new DFSAdmin command to set balancer bandwidth of
+ datanodes without restarting. (Eric Payne via szetszwo)
+
+ HDFS-2284. Add a new FileSystem, webhdfs://, for supporting write Http
+ access to HDFS. (szetszwo)
+
+ HDFS-2317. Support read access to HDFS in WebHDFS. (szetszwo)
+
+ HDFS-2338. Add configuration option to enable/disable WebHDFS.
+ (jitendra via szetszwo)
+
+ HDFS-2318. Provide authentication to WebHDFS using SPNEGO and delegation
+ tokens. (szetszwo)
+
+ HDFS-2340. Support getFileBlockLocations and getDelegationToken in WebHDFS.
+ (szetszwo)
+
+ HDFS-2348. Support getContentSummary and getFileChecksum in WebHDFS.
+ (szetszwo)
+
+ HDFS-2385. Support renew and cancel delegation tokens in WebHDFS.
+ (szetszwo)
+
+ HDFS-2539. Support doAs and GETHOMEDIRECTORY in WebHDFS.
+ (szetszwo)
+
+ IMPROVEMENTS
+
+ HDFS-1875. MiniDFSCluster hard-codes dfs.datanode.address to localhost
+ (Eric Payne via mattf)
+
+ HDFS-2019. Fix all the places where Java method File.list is used with
+ FileUtil.list API (Bharath Mundlapudi via mattf)
+
+ HDFS-1934. Fix NullPointerException when certain File APIs return null
+ (Bharath Mundlapudi via mattf)
+
+ HDFS-1510. Added test-patch.properties required by test-patch.sh (nigel)
+
+ HDFS-1628. Display full path in AccessControlException. (John George
+ via szetszwo)
+
+ HDFS-1707. Federation: Failure in browsing data on new namenodes.
+ (jitendra)
+
+ HDFS-1683. Test Balancer with multiple NameNodes. (szetszwo)
+
+ HDFS-1547. Improve decommission mechanism. (suresh)
+
+ HDFS-2143. Federation: In cluster web console, add link to namenode page
+ that displays live and dead datanodes. (Ravi Prakash via suresh)
+
+ HDFS-1588. Remove hardcoded strings for configuration keys, "dfs.hosts"
+ and "dfs.hosts.exlude". (Erik Steffl via suresh)
+
+ HDFS-1481. NameNode should validate fsimage before rolling. (hairong)
+
+ HDFS-1506. Refactor fsimage loading code. (hairong)
+
+ HDFS-1533. A more elegant FileSystem#listCorruptFileBlocks API
+ (HDFS portion) (Patrick Kling via hairong)
+
+ HDFS-1476. listCorruptFileBlocks should be functional while the
+ name node is in safe mode. (Patrick Kling via hairong)
+
+ HDFS-1534. Fix some incorrect logs in FSDirectory. (eli)
+
+ HDFS-1539. A config option for the datanode to fsycn a block file
+ when block is completely written. (dhruba)
+
+ HDFS-1335. HDFS side change of HADDOP-6904: RPC compatibility. (hairong)
+
+ HDFS-1557. Separate Storage from FSImage. (Ivan Kelly via jitendra)
+
+ HDFS-560 Enhancements/tuning to hadoop-hdfs/build.xml
+
+ HDFS-1629. Add a method to BlockPlacementPolicy for keeping the chosen
+ nodes in the output array. (szetszwo)
+
+ HDFS-1731. Allow using a file to exclude certain tests from build (todd)
+
+ HDFS-1736. Remove the dependency from DatanodeJspHelper to FsShell.
+ (Daryn Sharp via szetszwo)
+
+ HDFS-780. Revive TestFuseDFS. (eli)
+
+ HDFS-1445. Batch the calls in DataStorage to FileUtil.createHardLink().
+ (Matt Foley via jghoman)
+
+ HDFS-1763. Replace hard-coded option strings with variables from
+ DFSConfigKeys. (eli)
+
+ HDFS-1541. Not marking datanodes dead when namenode in safemode.
+ (hairong)
+
+ HDFS-1120. Make DataNode's block-to-device placement policy pluggable
+ (Harsh J Chouraria via todd)
+
+ HDFS-1785. In BlockReceiver and DataXceiver, clientName.length() is used
+ multiple times for determining whether the source is a client or a
+ datanode. (szetszwo)
+
+ HDFS-1789. Refactor frequently used codes from DFSOutputStream and
+ DataXceiver. (szetszwo)
+
+ HDFS-1767. Namenode ignores non-initial block report from datanodes
+ when in safemode during startup. (Matt Foley via suresh)
+
+ HDFS-1817. Move pipeline_Fi_[39-51] from TestFiDataTransferProtocol
+ to TestFiPipelineClose. (szetszwo)
+
+ HDFS-1760. In FSDirectory.getFullPathName(..), it is better to return "/"
+ for root directory instead of an empty string. (Daryn Sharp via szetszwo)
+
+ HDFS-1833. Reduce repeated string constructions and unnecessary fields,
+ and fix comments in BlockReceiver.PacketResponder. (szetszwo)
+
+ HDFS-1486. Generalize CLITest structure and interfaces to faciliate
+ upstream adoption (e.g. for web testing). (cos)
+
+ HDFS-1844. Move "fs -help" shell command tests from HDFS to COMMOM; see
+ also HADOOP-7230. (Daryn Sharp via szetszwo)
+
+ HDFS-1840. In DFSClient, terminate the lease renewing thread when all files
+ being written are closed for a grace period, and start a new thread when
+ new files are opened for write. (szetszwo)
+
+ HDFS-1854. make failure message more useful in
+ DFSTestUtil.waitReplication(). (Matt Foley via eli)
+
+ HDFS-1562. Add rack policy tests. (eli)
+
+ HDFS-1856. TestDatanodeBlockScanner waits forever, errs without giving
+ information. (Matt Foley via eli)
+
+ HDFS-1295. Improve namenode restart times by short-circuiting the
+ first block reports from datanodes. (Matt Foley via suresh)
+ Corrected merge error in DataNode.java. (Matt Foley)
+
+ HDFS-1843. Discover file not found early for file append.
+ (Bharath Mundlapudi via jitendra)
+
+ HDFS-1862. Improve test reliability of HDFS-1594. (Aaron T. Myers via eli)
+
+ HDFS-1846. Preallocate edit log with OP_INVALID instead of zero bytes
+ to ensure blocks are actually allocated. (Aaron T. Myers via todd)
+
+ HDFS-1741. Provide a minimal pom file to allow integration of HDFS into Sonar
+ analysis (cos)
+
+ HDFS-1870. Move and rename DFSClient.LeaseChecker to a seperated class
+ LeaseRenewer. (szetszwo)
+
+ HDFS-1866. Document dfs.datanode.max.transfer.threads in hdfs-default.xml
+ (Harsh J Chouraria via todd)
+
+ HDFS-1890. Improve the name, class and value type of the map
+ LeaseRenewer.pendingCreates. (szetszwo)
+
+ HDFS-1865. Share LeaseRenewer among DFSClients so that there is only a
+ LeaseRenewer thread per namenode per user. (szetszwo)
+
+ HDFS-1906. Remove logging exception stack trace in client logs when one of
+ the datanode targets to read from is not reachable. (suresh)
+
+ HDFS-1378. Edit log replay should track and report file offsets in case of
+ errors. (Aaron T. Myers and Todd Lipcon via todd)
+
+ HDFS-1917. Separate hdfs jars from common in ivy configuration. (Eric Yang
+ via szetszwo)
+
+ HDFS-1899. GenericTestUtils.formatNamenode should be moved to DFSTestUtil
+ (Ted Yu via todd)
+
+ HDFS-1117. Metrics 2.0 HDFS instrumentation. (Luke Lu via suresh)
+
+ HDFS-1946. HDFS part of HADOOP-7291. (eli)
+
+ HDFS-1945. Removed the deprecated fields in DataTransferProtocol.
+ (szetszwo)
+
+ HDFS-1730. Use DaemonFactory from common and delete it from HDFS.
+ (Tanping via suresh)
+
+ HDFS-1573. Add useful tracing information to Lease Renewer thread names
+ (todd)
+
+ HDFS-1939. In ivy.xml, test conf should not extend common conf.
+ (Eric Yang via szetszwo)
+
+ HDFS-1332 Include more information in exceptions and debug messages
+ when BlockPlacementPolicy cannot be satisfied. (Ted Yu via szetszwo)
+
+ HDFS-1958. Confirmation should be more lenient of user input when
+ formatting the NameNode. (todd)
+
+ HDFS-1905. Improve namenode -format command by not making -clusterId
+ parameter mandatory. (Bharath Mundlapudi via suresh)
+
+ HDFS-1877. Add a new test for concurrent read and write. (CW Chung
+ via szetszwo)
+
+ HDFS-1959. Better error message for missing namenode directory. (eli)
+
+ HDFS-1996. ivy: hdfs test jar should be independent to common test jar.
+ (Eric Yang via szetszwo)
+
+ HDFS-1812. TestHDFSCLI should clean up cluster in teardown method.
+ (Uma Maheswara Rao G via todd)
+
+ HDFS-1884. Improve TestDFSStorageStateRecovery to properly throw in the
+ case of errors. (Aaron T. Myers via todd)
+
+ HDFS-1727. fsck command should display command usage if user passes any
+ illegal argument. (Sravan Kumar via todd)
+
+ HDFS-1636. If dfs.name.dir points to an empty dir, namenode format
+ shouldn't require confirmation. (Harsh J Chouraria via todd)
+
+ HDFS-1966. Encapsulate individual DataTransferProtocol op headers.
+ (szetszwo)
+
+ HDFS-2024. Format TestWriteRead source codes. (CW Chung via szetszwo)
+
+ HDFS-1968. Enhance TestWriteRead to support position/sequential read,
+ append, truncate and verbose options. (CW Chung via szetszwo)
+
+ HDFS-1986. Add option to get http/https address from
+ DFSUtil#getInfoServer(). (Tanping via suresh)
+
+ HDFS-2029. In TestWriteRead, check visible length immediately after
+ openning the file and fix code style. (John George via szetszwo)
+
+ HDFS-2040. Only build libhdfs if a flag is passed. (eli)
+
+ HDFS-1586. Add InterfaceAudience and InterfaceStability annotations to
+ MiniDFSCluster. (suresh)
+
+ HDFS-2003. Separate FSEditLog reading logic from edit log memory state
+ building logic. (Ivan Kelly via todd)
+
+ HDFS-2066. Create a package and individual class files for
+ DataTransferProtocol. (szetszwo)
+
+ HADOOP-7106. Reorganize project SVN layout to "unsplit" the projects.
+ (todd, nigel)
+
+ HDFS-2046. Force entropy to come from non-true random for tests. (todd)
+
+ HDFS-2073. Add @Override annotation to NameNode. (suresh)
+
+ HDFS-420. Fuse-dfs should cache fs handles. (Brian Bockelman and eli)
+
+ HDFS-1568. Improve the log messages in DataXceiver. (Joey Echeverria via
+ szetszwo)
+
+ HDFS-2100. Improve TestStorageRestore. (atm)
+
+ HDFS-2092. Remove some object references to Configuration in DFSClient.
+ (Bharath Mundlapudi via szetszwo)
+
+ HDFS-2087. Declare methods in DataTransferProtocol interface, and change
+ Sender and Receiver to implement the interface. (szetszwo)
+
+ HDFS-1723. quota errors messages should use the same scale. (Jim Plush via
+ atm)
+
+ HDFS-2110. StreamFile and ByteRangeInputStream cleanup. (eli)
+
+ HDFS-2107. Move block management code from o.a.h.h.s.namenode to a new
+ package o.a.h.h.s.blockmanagement. (szetszwo)
+
+ HDFS-2109. Store uMask as member variable to DFSClient.Conf. (Bharath
+ Mundlapudi via szetszwo)
+
+ HDFS-2111. Add tests for ensuring that the DN will start with a few bad
+ data directories. (Harsh J Chouraria via todd)
+
+ HDFS-2134. Move DecommissionManager to the blockmanagement package.
+ (szetszwo)
+
+ HDFS-1977. Stop using StringUtils.stringifyException().
+ (Bharath Mundlapudi via jitendra)
+
+ HDFS-2131. Add new tests for the -overwrite/-f option in put and
+ copyFromLocal by HADOOP-7361. (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2140. Move Host2NodesMap to the blockmanagement package. (szetszwo)
+
+ HDFS-2154. In TestDFSShell, use TEST_ROOT_DIR and fix some deprecated
+ warnings. (szetszwo)
+
+ HDFS-2153. Move DFSClientAdapter to test and fix some javac warnings in
+ OfflineEditsViewerHelper. (szetszwo)
+
+ HDFS-2159. Deprecate DistributedFileSystem.getClient() and fixed the
+ deprecated warnings in DFSAdmin. (szetszwo)
+
+ HDFS-2157. Improve header comment in o.a.h.hdfs.server.namenode.NameNode.
+ (atm via eli)
+
+ HDFS-2147. Move cluster network topology to block management and fix some
+ javac warnings. (szetszwo)
+
+ HDFS-2141. Remove NameNode roles Active and Standby (they become
+ states of the namenode). (suresh)
+
+ HDFS-2161. Move createNamenode(..), createClientDatanodeProtocolProxy(..)
+ and Random object creation to DFSUtil; move DFSClient.stringifyToken(..)
+ to DelegationTokenIdentifier. (szetszwo)
+
+ HDFS-1774. Small optimization to FSDataset. (Uma Maheswara Rao G via eli)
+
+ HDFS-2167. Move dnsToSwitchMapping and hostsReader from FSNamesystem to
+ DatanodeManager. (szetszwo)
+
+ HDFS-2116. Use Mokito in TestStreamFile and TestByteRangeInputStream.
+ (Plamen Jeliazkov via shv)
+
+ HDFS-2112. Move ReplicationMonitor to block management. (Uma Maheswara
+ Rao G via szetszwo)
+
+ HDFS-1739. Add available volume size to the error message when datanode
+ throws DiskOutOfSpaceException. (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2144. If SNN shuts down during initialization it does not log the
+ cause. (Ravi Prakash via atm)
+
+ HDFS-2180. Refactor NameNode HTTP server into new class. (todd)
+
+ HDFS-2198. Remove hardcoded configuration keys. (suresh)
+
+ HDFS-2149. Move EditLogOp serialization formats into FsEditLogOp
+ implementations. (Ivan Kelly via todd)
+
+ HDFS-2191. Move datanodeMap from FSNamesystem to DatanodeManager.
+ (szetszwo)
+
+ HDFS-2200. Change FSNamesystem.LOG to package private. (szetszwo)
+
+ HDFS-2195. Refactor StorageDirectory to not be an non-static inner class.
+ (todd via eli)
+
+ HDFS-2212. Refactor double-buffering code out of EditLogOutputStreams.
+ (todd via eli)
+
+ HDFS-2199. Move blockTokenSecretManager from FSNamesystem to BlockManager.
+ (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2187. Make EditLogInputStream act like an iterator over FSEditLogOps
+ (Ivan Kelly and todd via todd)
+
+ HDFS-2225. Refactor edit log file management so it's not in classes
+ which should be generic to the type of edit log storage. (Ivan Kelly
+ via todd)
+
+ HDFS-2108. Move datanode heartbeat handling from namenode package to
+ blockmanagement package. (szetszwo)
+
+ HDFS-2226. Clean up counting of operations in FSEditLogLoader (todd)
+
+ HDFS-2228. Move block and datanode code from FSNamesystem to
+ BlockManager and DatanodeManager. (szetszwo)
+
+ HDFS-2238. In NamenodeFsck.toString(), uses StringBuilder.(..) instead of
+ string concatenation. (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2230. ivy to resolve/retrieve latest common-tests jar published by
+ hadoop common maven build. (gkesavan)
+
+ HDFS-2227. getRemoteEditLogManifest should pull its information from
+ FileJournalManager during checkpoint process (Ivan Kelly and Todd Lipcon
+ via todd)
+
+ HDFS-2239. Reduce access levels of the fields and methods in FSNamesystem.
+ (szetszwo)
+
+ HDFS-2241. Remove implementing FSConstants interface to just get the
+ constants from the interface. (suresh)
+
+ HDFS-2237. Change UnderReplicatedBlocks from public to package private.
+ (szetszwo)
+
+ HDFS-2233. Add WebUI tests with URI reserved chars. (eli)
+
+ HDFS-2265. Remove unnecessary BlockTokenSecretManager fields/methods from
+ BlockManager. (szetszwo)
+
+ HDFS-2260. Refactor BlockReader into an interface and implementation.
+ (todd)
+
+ HDFS-2096. Mavenization of hadoop-hdfs (Alejandro Abdelnur via tomwhite)
+
+ HDFS-2273. Refactor BlockManager.recentInvalidateSets to a new class.
+ (szetszwo)
+
+ HDFS-2266. Add Namesystem and SafeMode interfaces to avoid directly
+ referring to FSNamesystem in BlockManager. (szetszwo)
+
+ HDFS-1217. Change some NameNode methods from public to package private.
+ (Laxman via szetszwo)
+
+ HDFS-1620. Rename HdfsConstants -> HdfsServerConstants, FSConstants ->
+ HdfsConstants. (Harsh J Chouraria via atm)
+
+ HDFS-2197. Refactor RPC call implementations out of NameNode class (todd)
+
+ HDFS-2332. Add test for HADOOP-7629 (using an immutable FsPermission
+ object as an RPC parameter fails). (todd)
+
+ HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..)
+ to BlockManager. (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2209. Make MiniDFS easier to embed in other apps. (stevel)
+
+ HDFS-2205. Log message for failed connection to datanode is not
+ followed by a success message. (Ravi Prakash via stevel)
+
+ HDFS-2401. Running a set of methods in a Single Test Class.
+ (Jonathan Eagles via mahadev)
+
+ HDFS-2471. Add federation documentation. (suresh)
+
+ HDFS-2485. Improve code layout and constants in UnderReplicatedBlocks
+ (stevel)
+
+ HDFS-2356. Support case insensitive query parameter names in WebHDFS.
+ (szetszwo)
+
+ HDFS-2368. Move SPNEGO conf properties from hdfs-default.xml to
+ hdfs-site.xml. (szetszwo)
+
+ HDFS-2395. Add a root element in the JSON responses of WebHDFS.
+ (szetszwo)
+
+ HDFS-2427. Change the default permission in WebHDFS to 755 and add range
+ check/validation for all parameters. (szetszwo)
+
+ HDFS-2501. Add version prefix and root methods to WebHDFS. (szetszwo)
+
+ HDFS-1869. mkdirs should use the supplied permission for all of the created
+ directories. (Daryn Sharp via szetszwo)
+
+ HDFS-2355. Federation: enable using the same configuration file across
+ all the nodes in the cluster. (suresh)
+
+ HDFS-2371. Refactor BlockSender.java for better readability. (suresh)
+
+ HDFS-2493. Remove reference to FSNamesystem in blockmanagement classes.
+ (szetszwo)
+
+ HDFS-2294. Download of commons-daemon TAR should not be under target (tucu)
+
+ HDFS-2322. the build fails in Windows because commons-daemon TAR cannot be
+ fetched. (tucu)
+
+ HDFS-2436. Change FSNamesystem.setTimes(..) for allowing setting times on
+ directories. (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2512. Add textual error message to data transfer protocol responses
+ (todd)
+
+ HDFS-2521. Remove custom checksum headers from data transfer protocol
+ (todd)
+
+ HDFS-2308. NamenodeProtocol.endCheckpoint is vestigial and can be removed.
+ (eli)
+
+ HDFS-2507. Allow saveNamespace operations to be canceled. (todd)
+
+ OPTIMIZATIONS
+
+ HDFS-1458. Improve checkpoint performance by avoiding unnecessary image
+ downloads and loading. (hairong)
+
+ HDFS-1601. Pipeline ACKs are sent as lots of tiny TCP packets (todd)
+
+ HDFS-1826. NameNode should save image to name directories in parallel
+ during upgrade. (Matt Foley via hairong)
+
+ HDFS-2030. Improve usability of namenode -upgrade command.
+ (Bharath Mundlapudi via suresh)
+
+ HDFS-2056. Update fetchdt usage. (Tanping Wang via jitendra)
+
+ HDFS-2118. Couple dfs data dir improvements. (eli)
+
+ HDFS-2500. Avoid file system operations in BPOfferService thread while
+ processing deletes. (todd)
+
+ HDFS-2465. Add HDFS support for fadvise readahead and drop-behind. (todd)
+
+ BUG FIXES
+
+ HDFS-2344. Fix the TestOfflineEditsViewer test failure in 0.23 branch.
+ (Uma Maheswara Rao G via mattf)
+
+ HDFS-2347. Fix checkpointTxnCount's comment about editlog size.
+ (Uma Maheswara Rao G via mattf)
+
+ HDFS-2011. Removal and restoration of storage directories on checkpointing
+ failure doesn't work properly. (Ravi Prakash via mattf)
+
+ HDFS-1955. FSImage.doUpgrade() was made too fault-tolerant by HDFS-1826.
+ (mattf)
+
+ HDFS-2061. Two minor bugs in BlockManager block report processing. (mattf)
+
+ HDFS-1449. Fix test failures - ExtendedBlock must return
+ block file name in #getBlockName(). (suresh)
+
+ HDFS-1680. Fix TestBalancer. (szetszwo)
+
+ HDFS-1705. Balancer command throws NullPointerException. (suresh via
+ szetszwo)
+
+ HDFS-1559. Add missing UGM overrides to TestRefreshUserMappings
+ (Todd Lipcon via eli)
+
+ HDFS-1585. Fix build after HDFS-1547 (todd)
+
+ HDFS-1684. Balancer cannot start with with multiple namenodes. (szetszwo)
+
+ HDFS-1516. mvn-install is broken after 0.22 branch creation. (cos)
+
+ HDFS-1360. TestBlockRecovery should bind ephemeral ports.
+ (Todd Lipcon via hairong)
+
+ HDFS-1551. Fix pom templates dependency list (gkesavan)
+
+ HDFS-1509. A savenamespace command writes the fsimage and edits into
+ all configured directories. (dhruba)
+
+ HDFS-1540. Make Datanode handle errors from RPC calls to namenode
+ more elegantly. (dhruba)
+
+ HDFS-1463. Accesstime of a file is not updated in safeMode. (dhruba)
+
+ HDFS-863. Potential deadlock in TestOverReplicatedBlocks.
+ (Ken Goodhope via jghoman)
+
+ HDFS-1607. Fix referenced to misspelled method name getProtocolSigature
+ (todd)
+
+ HDFS-1610. Fix TestClientProtocolWithDelegationToken and TestBlockToken
+ on trunk after HADOOP-6904 (todd)
+
+ HDFS-1600. Fix release audit warnings on trunk. (todd)
+
+ HDFS-1691. Remove a duplicated static initializer for reading default
+ configurations in DFSck. (Alexey Diomin via szetszwo)
+
+ HDFS-1748. Balancer utilization classification is incomplete. (szetszwo)
+
+ HDFS-1738. change hdfs jmxget to return an empty string instead of
+ null when an attribute value is not available (tanping vi boryas)
+
+ HDFS-1757. Don't compile fuse-dfs by default. (eli)
+
+ HDFS-1770. TestFiRename fails due to invalid block size. (eli)
+
+ HDFS-1797. Fix new findbugs warning introduced by HDFS-1120 (todd)
+
+ HDFS-1611. Fix up some log messages in DFSClient and MBean registration
+ (Uma Maheswara Rao G via todd)
+
+ HDFS-1543. Reduce dev. cycle time by moving system testing artifacts from
+ default build and push to maven for HDFS (Luke Lu via cos)
+
+ HDFS-1818. TestHDFSCLI is failing on trunk after HADOOP-7202.
+ (Aaron T. Myers via todd)
+
+ HDFS-1828. TestBlocksWithNotEnoughRacks intermittently fails assert.
+ (Matt Foley via eli)
+
+ HDFS-1824. delay instantiation of file system object until it is
+ needed (linked to HADOOP-7207) (boryas)
+
+ HDFS-1831. Fix append bug in FileContext and implement CreateFlag
+ check (related to HADOOP-7223). (suresh)
+
+ HDFS-1594. When the disk becomes full Namenode is getting shutdown and
+ not able to recover. (Aaron T. Myers via eli)
+
+ HDFS-1822. Handle editlog opcode conflict with 0.20.203 during upgrade,
+ by throwing an error to indicate the editlog needs to be empty.
+ (suresh)
+
+ HDFS-1808. TestBalancer waits forever, errs without giving information.
+ (Matt Foley via eli)
+
+ HDFS-1829. TestNodeCount waits forever, errs without giving information.
+ (Matt Foley via eli)
+
+ HDFS-1860. when renewing/canceling DelegationToken over http we need to
+ pass exception information back to the caller.(boryas)
+
+ HDFS-1871. Mapreduce build fails due to MiniDFSCluster change from
+ HDFS-1052. (suresh)
+
+ HDFS-1876. One MiniDFSCluster constructor ignores numDataNodes parameter
+ (todd)
+
+ HDFS-1773. Do not show decommissioned datanodes, which are not in both
+ include and exclude lists, on web and JMX interfaces.
+ (Tanping Wang via szetszwo)
+
+ HDFS-1888. MiniDFSCluster#corruptBlockOnDatanodes() access must be
+ public. (suresh)
+
+ HDFS-1889. incorrect path in start/stop dfs script. (John George via eli)
+
+ HDFS-1891. Disable IPV6 for junit tests to fix TestBackupNode failure.
+ (suresh)
+
+ HDFS-1898. Tests failing on trunk due to use of NameNode.format.
+ (todd via eli)
+
+ HDFS-1902. Fix setrep path display for TestHDFSCLI. (Daryn Sharp
+ via szetszwo)
+
+ HDFS-1827. Fix timeout problem in TestBlockReplacement. (Matt Foley
+ via szetszwo)
+
+ HDFS-1908. Fix a NullPointerException in fi.DataTransferTestUtil.
+ (szetszwo)
+
+ HDFS-1912. Update tests for FsShell standardized error messages.
+ (Daryn Sharp via szetszwo)
+
+ HDFS-1903. Fix path display for rm/rmr in TestHDFSCLI and TestDFSShell.
+ (Daryn Sharp via szetszwo)
+
+ HDFS-1627. Fix NullPointerException in Secondary NameNode. (hairong)
+
+ HDFS-1928. Fix path display for touchz in TestHDFSCLI.
+ (Daryn Sharp via todd)
+
+ HDFS-1938. Fix ivy-retrieve-hdfs dependence in build.xml and aop.xml.
+ (Eric Yang via szetszwo)
+
+ HDFS-1929. TestEditLogFileOutputStream fails if running on same host as NN
+ (Aaron T. Myers via todd)
+
+ HDFS-1933. Update TestDFSShell for improved "test" shell command. (Daryn
+ Sharp via todd)
+
+ HDFS-1931. Update TestDFSShell for improved "du" shell command. (Daryn
+ Sharp via todd)
+
+ HDFS-1439. HDFS Federation: Fix compilation error in TestFiHftp. (suresh)
+
+ HDFS-1881. Federation: after taking snapshot the current directory
+ of datanode is empty. (Tanping Wang via suresh)
+
+ HDFS-1927. Fix a bug which causes ip=null in NameNode audit log.
+ (John George via szetszwo)
+
+ HDFS-1953. Federation: Change name node mxbean name in cluster web
+ console. (Tanping Wang via suresh)
+
+ HDFS-1922. Fix recurring failure of TestJMXGet (Luke Lu via todd)
+
+ HDFS-1371. One bad node can incorrectly flag many files as corrupt.
+ (Tanping Wang via jitendra)
+
+ HDFS-1943. Fail to start datanode while start-dfs.sh is executed by
+ root user. (Wei Yongjun via jghoman)
+
+ HDFS-1983. Fix path display for copy and rm commands in TestHDFSCLI and
+ TestDFSShell. (Daryn Sharp via todd)
+
+ HDFS-1999. Tests use deprecated configs. (Aaron T. Myers via eli)
+
+ HDFS-1592. Datanode startup doesn't honor volumes.tolerated.
+ (Bharath Mundlapudi via jitendra)
+
+ HDFS-1920. libhdfs does not build for ARM processors.
+ (Trevor Robinson via eli)
+
+ HDFS-1936. Layout version change from HDFS-1822 causes upgrade failure.
+ (suresh)
+
+ HDFS-2021. Update numBytesAcked before sending the ack in PacketResponder.
+ (John George via szetszwo)
+
+ HDFS-2020. Fix TestDFSUpgradeFromImage by removing the use of DataNode
+ as a singleton. (suresh via todd)
+
+ HDFS-2022. ant binary should build libhdfs. (Eric Yang via eli)
+
+ HDFS-2014. Change HDFS scripts to work in developer enviroment post
+ RPM packaging changes. (Eric Yang via suresh)
+
+ HDFS-1995. Federation: Minor bug fixes and modification cluster web UI.
+ (Tanping Wang via suresh)
+
+ HDFS-1907. Fix position read for reading still-being-written file in
+ DFSInputStream. (John George via szetszwo)
+
+ HDFS-1923. In TestFiDataTransferProtocol2, reduce random sleep time period
+ and increase the number of datanodes. (szetszwo)
+
+ HDFS-1149. Lease reassignment should be persisted to the edit log.
+ (Aaron T. Myers via todd)
+
+ HDFS-1998. Federation: Make refresh-namenodes.sh refresh all the
+ namenode. (Tanping Wang via suresh)
+
+ HDFS-2041. OP_CONCAT_DELETE doesn't properly restore modification time
+ of the concatenated file when edit logs are replayed. (todd)
+
+ HDFS-2063. libhdfs test is broken. (Eric Yang via eli)
+
+ HDFS-2067. Bump DATA_TRANSFER_VERSION constant in trunk after introduction
+ of protocol buffers in the protocol. (szetszwo via todd)
+
+ HDFS-2069. Incorrect default trash interval value in the docs.
+ (Harsh J Chouraria via eli)
+
+ HDFS-1942. Datanode must exist when all the block pool service threads
+ exit. (Bharath Mundlapudi via suresh)
+
+ HDFS-1656. Fixes an issue to do with fetching of delegation tokens in
+ HftpFileSystem. Contributed by Kan Zhang.
+
+ HDFS-1692. In secure mode, Datanode process doesn't exit when disks
+ fail. (Bharath Mundlapudi via suresh)
+
+ HDFS-1734. 'Chunk size to view' option is not working in Name Node UI.
+ (Uma Maheswara Rao G via jitendra)
+
+ HDFS-2086. If the include hosts list contains host names, after restarting
+ namenode, data nodes registration is denied. Contributed by Tanping Wang.
+
+ HDFS-2082. SecondaryNameNode web interface doesn't show the right info. (atm)
+
+ HDFS-1321. If service port and main port are the same, there is no clear
+ log message explaining the issue. (Jim Plush via atm)
+
+ HDFS-1381. HDFS javadocs hard-code references to dfs.namenode.name.dir and
+ dfs.datanode.data.dir parameters (Jim Plush via atm)
+
+ HDFS-2053. Bug in INodeDirectory#computeContentSummary warning.
+ (Michael Noll via eli)
+
+ HDFS-1990. Fix resource leaks in BlockReceiver.close(). (Uma Maheswara
+ Rao G via szetszwo)
+
+ HDFS-2034. Length in DFSInputStream.getBlockRange(..) becomes -ve when
+ reading only from a currently being written block. (John George via
+ szetszwo)
+
+ HDFS-2132. Potential resource leak in EditLogFileOutputStream.close. (atm)
+
+ HDFS-2120. on reconnect, DN can connect to NN even with different source
+ versions. (John George via atm)
+
+ HDFS-2152. TestWriteConfigurationToDFS causing the random failures. (Uma
+ Maheswara Rao G via atm)
+
+ HDFS-2114. re-commission of a decommissioned node does not delete
+ excess replicas. (John George via mattf)
+
+ HDFS-1776. Bug in Concat code. (Bharath Mundlapudi via Dmytro Molkov)
+
+ HDFS-2196. Make ant build system work with hadoop-common JAR generated
+ by Maven. (Alejandro Abdelnur via tomwhite)
+
+ HDFS-2245. Fix a NullPointerException in BlockManager.chooseTarget(..).
+ (szetszwo)
+
+ HDFS-2229. Fix a deadlock in namenode by enforcing lock acquisition
+ ordering. (szetszwo)
+
+ HDFS-2235. Encode servlet paths. (eli)
+
+ HDFS-2186. DN volume failures on startup are not counted. (eli)
+
+ HDFS-2240. Fix a deadlock in LeaseRenewer by enforcing lock acquisition
+ ordering. (szetszwo)
+
+ HDFS-73. DFSOutputStream does not close all the sockets.
+ (Uma Maheswara Rao G via eli)
+
+ HDFS-1257. Fix a race condition on BlockManager.recentInvalidateSets.
+ (Eric Payne via szetszwo)
+
+ HDFS-2267. DataXceiver thread name incorrect while waiting on op during
+ keepalive. (todd)
+
+ HDFS-1480. All replicas of a block can end up on the same rack when
+ some datanodes are decommissioning. (todd)
+
+ HDFS-2286. DataXceiverServer logs AsynchronousCloseException at shutdown
+ (todd)
+
+ HDFS-2289. Ensure jsvc is bundled with the HDFS distribution artifact.
+ (Alejandro Abdelnur via acmurthy)
+
+ HDFS-2314. MRV1 test compilation broken after HDFS-2197 (todd)
+
+ HDFS-2323. start-dfs.sh script fails for tarball install (tomwhite)
+
+ HDFS-2346. TestHost2NodesMap & TestReplicasMap will fail depending upon
+ execution order of test methods (Laxman via atm)
+
+ HDFS-2345. TestLeaseRecovery2 fails on 0.23 branch (Uma Maheswara Rao G
+ via atm)
+
+ HDFS-2412. Add backwards-compatibility layer for renamed FSConstants
+ class (todd)
+
+ HDFS-2414. Fix TestDFSRollback to avoid spurious failures. (todd)
+
+ HDFS-2422. The NN should tolerate the same number of low-resource volumes
+ as failed volumes (atm)
+
+ HDFS-2467. HftpFileSystem uses incorrect compare for finding delegation
+ tokens. (omalley)
+
+ HDFS-2331. Fix WebHdfsFileSystem compilation problems for a bug in JDK
+ version < 1.6.0_26. (Abhijit Suresh Shingate via szetszwo)
+
+ HDFS-2333. Change DFSOutputStream back to package private, otherwise,
+ there are two SC_START_IN_CTOR findbugs warnings. (szetszwo)
+
+ HDFS-2366. Initialize WebHdfsFileSystem.ugi in object construction.
+ (szetszwo)
+
+ HDFS-2361. hftp is broken, fixed username checks in JspHelper. (jitendra)
+
+ HDFS-2403. NamenodeWebHdfsMethods.generateDelegationToken(..) does not use
+ the renewer parameter. (szetszwo)
+
+ HDFS-2409. _HOST in dfs.web.authentication.kerberos.principal. (jitendra)
+
+ HDFS-2404. WebHDFS liststatus json response is not correct. (suresh)
+
+ HDFS-2441. Remove the Content-Type set by HttpServer.QuotingInputFilter in
+ WebHDFS responses. (szetszwo)
+
+ HDFS-2428. Convert com.sun.jersey.api.ParamException$QueryParamException
+ to IllegalArgumentException and response it as http BAD_REQUEST in WebHDFS.
+ (szetszwo)
+
+ HDFS-2424. Added a root element "HdfsFileStatuses" for the response
+ of WebHDFS listStatus. (szetszwo)
+
+ MAPREDUCE-2764. Fix renewal of dfs delegation tokens. (Owen via jitendra)
+
+ HDFS-2439. Fix NullPointerException in WebHDFS when opening a non-existing
+ file or creating a file without specifying the replication parameter.
+ (szetszwo)
+
+ HDFS-2453. Fix http response code for partial content in WebHDFS, added
+ getDefaultBlockSize() and getDefaultReplication() in WebHdfsFileSystem
+ and cleared content type in ExceptionHandler. (szetszwo)
+
+ HDFS-2411. The the auth to local mappings are not being respected, with
+ WebHDFS enabled. (jitendra)
+
+ HDFS-2494. Close the streams and DFSClient in DatanodeWebHdfsMethods.
+ (Uma Maheswara Rao G via szetszwo)
+
+ HDFS-2298. Fix TestDfsOverAvroRpc by changing ClientProtocol to
+ not include multiple methods of the same name. (cutting)
+
+ HDFS-2432. WebHDFS: response FORBIDDEN when setReplication on non-files;
+ clear umask before creating a flie; throw IllegalArgumentException if
+ setOwner with both owner and group empty; throw FileNotFoundException if
+ getFileStatus on non-existing files; fix bugs in getBlockLocations; and
+ changed getFileChecksum json response root to "FileChecksum". (szetszwo)
+
+ HDFS-2065. Add null checks in DFSClient.getFileChecksum(..). (Uma
+ Maheswara Rao G via szetszwo)
+
+ HDFS-2522. Disable TestDfsOverAvroRpc test. (suresh)
+
+ HDFS-2416. distcp with a WebHDFS uri on a secure cluster fails. (jitendra)
+
+ HDFS-2527. WebHDFS: remove the use of "Range" header in Open; use ugi
+ username if renewer parameter is null in GetDelegationToken; response OK
+ when setting replication for non-files; rename GETFILEBLOCKLOCATIONS to
+ GET_BLOCK_LOCATIONS and state that it is a private unstable API; replace
+ isDirectory and isSymlink with enum {FILE, DIRECTORY, SYMLINK} in
+ HdfsFileStatus JSON object. (szetszwo)
+
+ HDFS-2528. WebHDFS: set delegation kind to WEBHDFS and add a HDFS token
+ when http requests are redirected to datanode. (szetszwo)
+
+ HDFS-2540. WebHDFS: change "Expect: 100-continue" to two-step write; change
+ "HdfsFileStatus" and "localName" respectively to "FileStatus" and
+ "pathSuffix" in JSON response. (szetszwo)
+
+ BREAKDOWN OF HDFS-1073 SUBTASKS
+
+ HDFS-1521. Persist transaction ID on disk between NN restarts.
+ (Ivan Kelly and Todd Lipcon via todd)
+
+ HDFS-1538. Refactor more startup and image loading code out of FSImage.
+ (todd)
+
+ HDFS-1729. Add code to detect valid length of an edits file. (todd)
+
+ HDFS-1793. Add code to inspect a storage directory with txid-based
+ filenames (todd)
+
+ HDFS-1794. Add code to list which edit logs are available on a remote NN
+ (todd)
+
+ HDFS-1858. Add state management variables to FSEditLog (Ivan Kelly and Todd
+ Lipcon via todd)
+
+ HDFS-1859. Add some convenience functions to iterate over edit log streams
+ (Ivan Kelly and Todd Lipcon via todd)
+
+ HDFS-1894. Add constants for LAYOUT_VERSIONs in edits log branch (todd)
+
+ HDFS-1892. Fix EditLogFileInputStream.getValidLength to be aware of
+ OP_INVALID filler (todd)
+
+ HDFS-1799. Refactor log rolling and filename management out of FSEditLog
+ (Ivan Kelly and Todd Lipcon via todd)
+
+ HDFS-1801. Remove use of timestamps to identify checkpoints and logs (todd)
+
+ HDFS-1930. TestDFSUpgrade failing in HDFS-1073 branch (todd)
+
+ HDFS-1800. Extend image checksumming to function with multiple fsimage
+ files per directory. (todd)
+
+ HDFS-1725. Set storage directories only at FSImage construction (Ivan Kelly
+ via todd)
+
+ HDFS-1926. Remove references to StorageDirectory from JournalManager
+ interface (Ivan Kelly via todd)
+
+ HDFS-1893. Change edit logs and images to be named based on txid (todd)
+
+ HDFS-1985. Clean up image transfer servlet (todd)
+
+ HDFS-1984. Enable multiple secondary namenodes to run simultaneously (todd)
+
+ HDFS-1987. Re-enable TestCheckpoint.testSecondaryImageDownload which was
+ not running previously. (todd)
+
+ HDFS-1993. TestCheckpoint needs to clean up between cases (todd)
+
+ HDFS-1992. Remove vestiges of NNStorageListener. (todd)
+
+ HDFS-1991. Some refactoring of Secondary NameNode to be able to share more
+ code with the BackupNode or CheckpointNode. (todd)
+
+ HDFS-1994. Fix race conditions when running two rapidly checkpointing
+ Secondary NameNodes. (todd)
+
+ HDFS-2001. Remove use of previous.checkpoint and lastcheckpoint.tmp
+ directories (todd)
+
+ HDFS-2015. Remove checkpointTxId from VERSION file. (todd)
+
+ HDFS-2016. Add infrastructure to remove or archive old and unneeded storage
+ files within the name directories. (todd)
+
+ HDFS-2047. Improve TestNamespace and TestEditLog in HDFS-1073 branch.
+ (todd)
+
+ HDFS-2048. Add upgrade tests and fix upgrade from 0.22 with corrupt image.
+ (todd)
+
+ HDFS-2027. Image inspector should return finalized logs before unfinalized
+ logs. (todd)
+
+ HDFS-2074. Determine edit log validity by truly reading and validating
+ transactions. (todd)
+
+ HDFS-2085. Finalize in-progress edit logs at startup. (todd)
+
+ HDFS-2026. SecondaryNameNode should properly handle the case where the
+ NameNode is reformatted. (todd)
+
+ HDFS-2077. Address checkpoint upload when one of the storage dirs is failed
+ (todd)
+
+ HDFS-2078. NameNode should not clear directory when restoring removed
+ storage. (todd)
+
+ HDFS-2088. Move edits log archiving logic into FSEditLog/JournalManager
+ (todd)
+
+ HDFS-2093. Handle case where an entirely empty log is left during NN crash
+ (todd)
+
+ HDFS-2102. Zero-pad edits filename to make them lexically sortable. (Ivan
+ Kelly via todd)
+
+ HDFS-2010. Fix NameNode to exit if all edit streams become inaccessible.
+ (atm via todd)
+
+ HDFS-2123. Checkpoint interval should be based on txn count, not size.
+ (todd)
+
+ HDFS-1979. Fix backupnode for new edits/image layout. (todd)
+
+ HDFS-2101. Fix remaining unit tests for new storage filenames. (todd)
+
+ HDFS-2133. Address remaining TODOs and pre-merge cleanup on HDFS-1073
+ branch. (todd)
+
+ HDFS-1780. Reduce need to rewrite FSImage on startup. (todd)
+
+ HDFS-2104. Add a flag to the 2NN to format its checkpoint dirs on startup.
+ (todd)
+
+ HDFS-2135. Fix regression of HDFS-1955 in HDFS-1073 branch. (todd)
+
+ HDFS-2160. Fix CreateEditsLog test tool in HDFS-1073 branch. (todd)
+
+ HDFS-2168. Reenable TestEditLog.testFailedOpen and fix exposed bug. (todd)
+
+ HDFS-2169. Clean up TestCheckpoint and remove TODOs (todd)
+
+ HDFS-2170. Address remaining TODOs in HDFS-1073 branch. (todd)
+
+ HDFS-2172. Address findbugs and javadoc warnings in HDFS-1073 branch.
+ (todd)
+
+ HDFS-2445. Ensure failed tests exit with proper error code. (Jonathan
+ Eagles via acmurthy)
+
+Release 0.22.1 - Unreleased
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ HDFS-2718. Optimize OP_ADD in edits loading. (shv)
+
+ HDFS-2886. CreateEditLogs should generate a realistic edit log. (shv)
+
+ BUG FIXES
+
+ HDFS-2877. If locking of a storage dir fails, it will remove the other
+ NN's lock file on exit. (todd)
+
+Release 0.22.0 - 2011-11-29
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-1825. Remove thriftfs contrib. (nigel via eli)
+
+ NEW FEATURES
+
+ HDFS-992. Re-factor block access token implementation to conform to the
+ generic Token interface in Common (Kan Zhang and Jitendra Pandey via jghoman)
+
+ HDFS-599. Allow NameNode to have a seprate port for service requests from
+ client requests. (Dmytro Molkov via hairong)
+
+ HDFS-1004. Update NN to support Kerberized SSL from HADOOP-6584.
+ (jghoman and Kan Zhang via jghoman)
+
+ HDFS-1005. Fsck security. (borya and Kan Zhang via jghoman)
+
+ HDFS-1006. getImage/putImage http requests should be https for the case
+ of security enabled. (borya and jghoman via jghoman)
+
+ HDFS-1033. In secure clusters, NN and SNN should verify that the remote
+ principal during image and edits transfer. (jghoman)
+
+ HDFS-1023. Allow http server to start as regular principal if https
+ principal not defined. (jghoman)
+
+ HDFS-1150. Verify datanodes' identities to clients in secure clusters.
+ (jghoman)
+
+ HDFS-1330. Make RPCs to DataNodes timeout. (hairong)
+ Added additional unit tests per HADOOP-6889. (John George via mattf)
+
+ HDFS-202. HDFS support of listLocatedStatus introduced in HADOOP-6870.
+ HDFS piggyback block locations to each file status when listing a
+ directory. (hairong)
+
+ HDFS-1361. Add -fileStatus operation to NNThroughputBenchmark. (shv)
+
+ HDFS-1435. Provide an option to store fsimage compressed. (hairong)
+
+ HDFS-903. Support fsimage validation through MD5 checksum. (hairong)
+
+ HDFS-1457. Provide an option to throttle image transmission between
+ pimary and secondary NameNodes. (Yifei Lu and hairong via hairong)
+
+ HDFS-1164. TestHdfsProxy is failing. (Todd Lipcon via cos)
+
+ HDFS-811. Add metrics, failure reporting and additional tests for HDFS-457.
+ (eli)
+
+ HDFS-895. Allow hflush/sync to occur in parallel with new writes
+ to the file. (Todd Lipcon via hairong)
+
+ HDFS-528. Add ability for safemode to wait for a minimum number of
+ live datanodes (Todd Lipcon via eli)
+
+ HDFS-1753. Resource Leak in StreamFile. (Uma Maheswara Rao G via eli)
+
+ IMPROVEMENTS
+
+ HDFS-1304. Add a new unit test for HftpFileSystem.open(..). (szetszwo)
+
+ HDFS-1096. fix for prev. commit. (boryas)
+
+ HDFS-1096. allow dfsadmin/mradmin refresh of superuser proxy group
+ mappings (boryas)
+
+ HDFS-1146. Javadoc for getDelegationTokenSecretManager in FSNamesystem (jnp via boryas)
+
+ HDFS-1132. Refactor TestFileStatus (Eli Collins via cos)
+
+ HDFS-1163. normalize property names for JT/NN kerberos principal
+ names in configuration (from HADOOP 6633) (boryas)
+
+ HDFS-1003. authorization checks for inter-server protocol
+ (based on HADOOP-6600) (boryas)
+
+ HDFS-1061. Memory footprint optimization for INodeFile object.
+ (Bharath Mundlapudi via jghoman)
+
+ HDFS-1079. Throw exceptions as specified by the AbstractFileSystem
+ in HDFS implemenation and protocols. (suresh)
+
+ HDFS-1112. Edit log buffer should not grow unfoundedly. (hairong)
+
+ HDFS-1119. Introduce a GSet interface to BlocksMap. (szetszwo)
+
+ HDFS-1184. Replace tabs in code with spaces. (Jeff Ames via jghoman)
+
+ HDFS-1185. Remove duplicate now() functions in DataNode, FSNamesysetm.
+ (Jeff Ames via jghoman)
+
+ HDFS-1183. Remove some duplicate code in NamenodeJspHelper.java.
+ (Jeff Ames via jghoman)
+
+ HDFS-1190. Remove unused getNamenode() method from DataNode.
+ (Jeff Ames via jghoman)
+
+ HDFS-1110. Reuses objects for commonly used file names in namenode to
+ reduce the heap usage. (suresh)
+
+ HDFS-752. Add interfaces classification to to HDFS source code. (suresh)
+
+ HDFS-947. An Hftp read request is redirected to a datanode that has
+ the most replicas of the blocks in the file. (Dmytro Molkov via dhruba)
+
+ HDFS-1272. Fixes to take care of the changes in HADOOP-6845.
+ (Jitendra Pandey via ddas)
+
+ HDFS-1298 - Add support in HDFS for new statistics added in FileSystem
+ to track the file system operations. (suresh)
+
+ HDFS-1201. The HDFS component for HADOOP-6632.
+ (Kan Zhang & Jitendra Pandey via ddas)
+
+ HDFS-1307 Add start time, end time and total time taken for FSCK to
+ FSCK report (suresh)
+
+ HDFS-1302. The HDFS side of the changes corresponding to HADOOP-6861.
+ (Jitendra Pandey & Owen O'Malley via ddas)
+
+ HDFS-1315. Add fsck event to audit log and remove other audit log events
+ corresponding to FSCK listStatus and open calls. (suresh)
+
+ HDFS-1178. The NameNode servlets should not use RPC to connect to the
+ NameNode. (Kan Zhang via jghoman)
+
+ HDFS-1130. Adds dfs.cluster.administrator ACL configuration that can
+ be used to control who can view the default hdfs servlets. (ddas)
+
+ HDFS-1297. Fix some comments. (Jeff Ames via jghoman)
+
+ HDFS-330. Datanode Web UIs should provide robots.txt.
+ (Allen Wittenauer via jghoman)
+
+ HDFS-881. Refactor DataNode Packet header into DataTransferProtocol.
+ (Todd Lipcon via jghoman)
+
+ HDFS-1036. docs for fetchdt
+
+ HDFS-1318. Add JMX interface for read access to namenode and datanode
+ web UI information. (Tanping Wang via suresh).
+
+ HDFS-1356. Provide information as to whether or not security is
+ enabled on web interface for NameNode (boryas)
+
+ HDFS-1205. FSDatasetAsyncDiskService should name its threads.
+ (Todd Lipcon via eli)
+
+ HDFS-1111. Introduce getCorruptFileBlocks() for fsck. (Sriram Rao via shv)
+
+ HDFS-1395. Add @Override to FSDataset methods that implement
+ FSDatasetInterface methods. (suresh)
+
+ HDFS-1383. Improve the error messages when using hftp://. (szetszwo)
+
+ HDFS-1093. Change the FSNamesystem lock to a read/write lock. (dhruba)
+
+ HDFS-1407. Change DataTransferProtocol methods to use Block instead
+ of individual elements of Block. (suresh)
+
+ HDFS-1417. Add @Override to SimulatedFSDataset methods that implement
+ FSDatasetInterface methods. (suresh)
+
+ HDFS-1426. Remove unused method BlockInfo#listCount. (hairong)
+
+ HDFS-1472. Allow programmatic access to fsck output.
+ (Ramkumar Vadali via dhruba)
+
+ HADOOP-7007. Update the hudson-test-patch ant target to work with the
+ latest test-patch.sh script (gkesavan)
+
+ HDFS-1462. Refactor edit log loading to a separate class from edit log writing.
+ (Todd Lipcon via eli)
+
+ HDFS-1485. Fix typo in BlockPlacementPolicy. (Jingguo Yao via shv)
+
+ HDFS-1035. Generate Eclipse's .classpath file from Ivy config. (nigel)
+
+ HDFS-1408. Herriot NN and DN clients should vend statistics. (cos)
+
+ HDFS-1491 Update Hdfs to match the change of methods from protected to public
+ in AbstractFileSystem (Hadoop-6903) (sanjay)
+
+ HDFS-1160. Improve some FSDataset warnings and comments. (eli)
+
+ HDFS-556. Provide info on failed volumes in the web ui. (eli)
+
+ HDFS-697. Enable asserts for tests by default. (eli)
+
+ HDFS-1187. Modify fetchdt to allow renewing and canceling token.
+ (Owen O'Malley and Kan Zhang via jghoman)
+
+ HDFS-1387. Update HDFS permissions guide for security. (Todd Lipcon via eli)
+
+ HDFS-455. Make NN and DN handle in a intuitive way comma-separated
+ configuration strings. (Michele Catasta via eli)
+
+ HDFS-1071. savenamespace should write the fsimage to all configured
+ fs.name.dir in parallel (Dmytro Molkov via jghoman)
+
+ HDFS-1055. Improve thread naming for DataXceivers.
+ (Todd Lipcon and Ramkumar Vadali via eli).
+
+ HDFS-718. Configuration parameter to prevent accidental formatting of
+ HDFS filesystem. (Andrew Ryan via jghoman)
+
+ HDFS-1500. TestOfflineImageViewer failing on trunk. (Todd Lipcon
+ via hairong)
+
+ HDFS-1483. DFSClient.getBlockLocations should indicate if corresponding
+ blocks are corrupt. (Patrick Kling via hairong)
+
+ HDFS-259. Remove intentionally corrupt 0.13 directory layout creation.
+ (Todd Lipcon via eli)
+
+ HDFS-1513. Fix a number of warnings. (eli)
+
+ HDFS-1473. Refactor storage management into separate classes than fsimage
+ file reading/writing. (Todd Lipcon via eli)
+
+ HDFS-1582. Remove auto-generated native build files. (rvs via eli)
+
+ HDFS-1456. Provide builder for constructing instances of MiniDFSCluster.
+ (jghoman)
+
+ HDFS-1861. Rename dfs.datanode.max.xcievers and bump its default value.
+ (eli)
+
+ HDFS-1052. HDFS Federation - Merge of umbrella jira changes from
+ HDFS-1052 branch into trunk.
+
+ HDFS-1835. DataNode should not depend on SHA1PRNG secure random generator
+ to generate a storage ID. (John Carrino via todd)
+
+ HDFS-1947. DFSClient should use mapreduce.task.attempt.id. (eli)
+
+ HDFS-1957. Add documentation for HFTP. (Ari Rabkin via todd)
+
+ HDFS-1454. Update the documentation to reflect that clients don't write
+ blocks to local disk before copying to HDFS. (Harsh J Chouraria via todd)
+
+ HDFS-1980. Move build/webapps deeper in the build directory heirarchy
+ to aid eclipse users. (todd)
+
+ HDFS-1619. Remove AC_TYPE* from the libhdfs. (Roman Shaposhnik via eli)
+
+ HDFS-1948 Forward port 'hdfs-1520 lightweight namenode operation to
+ trigger lease recovery' (stack)
+
+ HDFS-1954. Improved corrupt files warning on NameNode web UI.
+ (Patrick Hunt via shv)
+
+ HDFS-1409. BackupNode registration throwing
+ UnsupportedActionException("register") instead of "journal".
+ (Ching-Shen Chen via shv)
+
+ HDFS-2054 BlockSender.sendChunk() prints ERROR for connection closures
+ encountered during transferToFully() (Kihwal Lee via stack)
+
+ OPTIMIZATIONS
+
+ HDFS-1140. Speedup INode.getPathComponents. (Dmytro Molkov via shv)
+
+ HDFS-1081. Performance regression in
+ DistributedFileSystem::getFileBlockLocations in secure systems (jghoman)
+
+ HDFS-1114. Implement LightWeightGSet for BlocksMap in order to reduce
+ NameNode memory footprint. (szetszwo)
+
+ HDFS-1320. Add LOG.isDebugEnabled() guard for each LOG.debug(..).
+ (Erik Steffl via szetszwo)
+
+ HDFS-1368. Add a block counter to DatanodeDescriptor. (hairong)
+
+ HDFS-1434. Refactor Datanode#startDataNode method into smaller methods.
+ (suresh)
+
+ HDFS-941. The DFS client should cache and reuse open sockets to datanodes
+ while performing reads. (bc Wong and Todd Lipcon via todd)
+
+ BUG FIXES
+
+ HDFS-1039. Adding test for JspHelper.getUGI(jnp via boryas)
+
+ HDFS-1019. Incorrect default values for delegation tokens in
+ hdfs-default.xml (jnp via boryas)
+
+ HDFS-1039. Service should be set in the token in JspHelper.getUGI(jnp via boryas)
+
+ HDFS-1038. FIX. A test missed in a previous commit for this JIRA. (boryas)
+
+ HDFS-1038. In nn_browsedfscontent.jsp fetch delegation token only
+ if security is enabled. (jnp via boryas)
+
+ HDFS-1044. Cannot submit mapreduce job from secure client to
+ unsecure sever (boryas)
+
+ HDFS-1021. specify correct server principal for RefreshAuthorizationPolicyProtocol
+ and RefreshUserToGroupMappingsProtocol protocols in DFSAdmin (for HADOOP-6612) (boryas)
+
+ HDFS-970. fsync fsimage to disk before closing fsimage file.
+ (Todd Lipcon via dhruba)
+
+ HDFS-1027. Update copyright year to 2010. (Ravi Phulari via jghoman)
+
+ HDFS-1080. SecondaryNameNode image transfer should use the defined http
+ address rather than local ip address. (jghoman)
+
+ HDFS-1198. Resolving cross-realm principals. (Jitendra Pandey via jghoman)
+
+ HDFS-1118. Fix socketleak on DFSClient. (Zheng Shao via dhruba)
+
+ HDFS-1192. refreshSuperUserGroupsConfiguration should use server side
+ configuration for the refresh (for HADOOP-6815) (boryas)
+
+ HDFS-1036. in DelegationTokenFetch dfs.getURI returns no port (boryas)
+
+ HDFS-1017. browsedfs jsp should call JspHelper.getUGI rather
+ than using createRemoteUser() (jnp via boryas)
+
+ HDFS-1250. Namenode should reject block reports and block received
+ requests from dead datanodes (suresh)
+
+ HDFS-1145. When NameNode is shutdown it does not try to exit
+ safemode anymore. (dhruba)
+
+ HDFS-1202. DataBlockScanner throws NPE when updated before
+ initialized. (Todd Lipcon via dhruba)
+
+ HDFS-882. Datanode logs the hostname and port its listening on.
+ (Steve Loughran via dhruba)
+
+ HDFS-1238. ant eclipse-files has drifted again, (jghoman)
+
+ HDFS-1045. In secure clusters, re-login is necessary for https
+ clients before opening connections. (jghoman)
+
+ HDFS-1289. Datanode secure mode is broken. (Kan Zhang via jghoman)
+
+ HDFS-1007. HFTP needs to be updated to use delegation tokens (boryas)
+
+ HDFS-1085. HFTP read may fail silently on the client side if there is an
+ exception on the server side. (szetszwo)
+
+ HDFS-1308. job conf key for the services name of DelegationToken for HFTP
+ url is constructed incorrectly in HFTPFileSystem (boryas)
+
+ HDFS-1319. Fix location of re-login for secondary namenode from HDFS-999.
+ (jghoman)
+
+ HDFS-1317. Remove the FILEPATH_PATTERN from hdfsproxy.AuthorizationFilter.
+ (Rohini Palaniswamy via szetszwo)
+
+ HDFS-912. sed in build.xml on Solaris fails. (Allen Wittenauer via jghoman)
+
+ HDFS-1296. using delegation token over hftp for long running
+ clients (boryas)
+
+ HDFS-1334. open in HftpFileSystem does not add delegation tokens to the url.
+ (Jitendra Pandey via jghoman)
+
+ HDFS-1301. TestHDFSProxy need to use server side conf for ProxyUser
+ stuff. (boryas)
+
+ HDFS-1340. When security is turned off, there is a potential XSS attack.
+ This patch fixes it by removing delegationtoken string from the URL,
+ before returning a response to the client. (Jitendra Pandey via ddas)
+
+ HDFS-1347. TestDelegationToken uses mortbay.log for logging (boryas)
+
+ HDFS-1157. Modifications introduced by HDFS-1150 are breaking aspect's
+ bindings (cos)
+
+ HDFS-1349. Remove empty java files. (Eli Collins)
+
+ HDFS-1340. A null delegation token is appended to the url if security
+ is disabled when browsing filesystem. (boryas)
+
+ HDFS-1352. Fix jsvc.location. (Eli Collins via jghoman)
+
+ HDFS-1284. TestBlockToken fails. (Kan Zhang via jghoman)
+
+ HDFS-1355. ant veryclean (clean-cache) doesn't clean enough.
+ (Luke Lu via jghoman)
+
+ HDFS-1353. Remove most of getBlockLocation optimization. (jghoman)
+
+ HDFS-1369. Invalid javadoc reference in FSDatasetMBean.java (Eli Collins)
+
+ HDFS-829. hdfsJniHelper.c: #include <error.h> is not portable.
+ (Allen Wittenauer via jghoman)
+
+ HDFS-1310. The ClientDatanodeProtocol proxy should be stopped in
+ DFSInputStream.readBlockLength(..). (sam rash via szetszwo)
+
+ HDFS-1357. HFTP traffic served by DataNode shouldn't use service port
+ on NameNode. (Kan Zhang via jghoman)
+
+ HDFS-1419. HDFS Federation: Three test cases need minor modification after
+ the new block id change (Tanping Wang via suresh)
+
+ HDFS-96. HDFS supports blocks larger than 2 GB.
+ (Patrick Kling via dhruba)
+
+ HDFS-1433. Fix test failures - TestPread and TestFileLimit. (suresh)
+
+ HDFS-1364. Makes long running HFTP-based applications do relogins
+ if necessary. (Jitendra Pandey via ddas)
+
+ HDFS-1399. Distinct minicluster services (e.g. NN and JT) overwrite each
+ other's service policies. (Aaron T. Myers via tomwhite)
+
+ HDFS-1440. Fix TestComputeInvalidateWork failure. (suresh)
+
+ HDFS-1498. FSDirectory#unprotectedConcat calls setModificationTime
+ on a file. (eli)
+
+ HDFS-1625. Ignore disk space values in TestDataNodeMXBean. (szetszwo)
+
+ HDFS-1850. DN should transmit absolute failed volume count rather than
+ increments to the NN. (eli)
+
+ HDFS-671. Documentation change for updated configuration keys.
+ (tomwhite via eli)
+
+ HDFS-1544. Ivy resolve force mode should be turned off by default.
+ (Luke Lu via tomwhite)
+
+ HDFS-1615. seek() on closed DFS input stream throws NullPointerException
+ (Scott Carey via todd)
+
+ HDFS-1897. Documentation refers to removed option dfs.network.script
+ (Andrew Whang via todd)
+
+ HDFS-1621. Fix references to hadoop-common-${version} in build.xml
+ (Jolly Chen via todd)
+
+ HDFS-1505. saveNamespace appears to succeed even if all directories fail
+ to save. (Aaron T. Myers via todd)
+
+ HDFS-1921. saveNamespace can cause NN to be unable to come up on restart
+ (Matt Foley via todd)
+
+ HDFS-1925. SafeModeInfo should use the correct constant instead of a
+ hard-coded value for its default. (Joey Echeverria via todd)
+
+ HDFS-1575. Viewing block from web UI is broken. (Aaron T. Myers via todd)
+
+ HDFS-1932. Ensure that HDFS configuration deprecations are set up in every
+ spot that HDFS configurations are loaded. (Jolly Chen via todd)
+
+ HDFS-1952. FSEditLog.open() appears to succeed even if all EDITS
+ directories fail. (Andrew Wang via todd)
+
+ HDFS-1965. IPCs done using block token-based tickets can't reuse
+ connections (todd)
+
+ HDFS-1978. All but first option in LIBHDFS_OPTS is ignored. (eli)
+
+ HDFS-1964. Fix incorrect HTML unescaping in DatanodeJspHelper
+ (Aaron T. Myers via todd)
+
+ HDFS-1997. Image transfer process misreports client side exceptions.
+ (todd via eli)
+
+ HDFS-2000. Missing deprecation for io.bytes.per.checksum.
+ (Aaron T. Myers vie eli)
+
+ HDFS-977. DataNode.createInterDataNodeProtocolProxy() guards a log
+ at the wrong level. (Harsh J Chouraria via todd)
+
+ HDFS-1969. Running rollback on new-version namenode destroys the
+ namespace. (todd)
+
+ HDFS-2039. TestNameNodeMetrics uses a bad test root path, preventing it
+ from running inside Eclipse. (todd)
+
+ HDFS-988. saveNamespace race can corrupt the edits log. (eli)
+
+ HDFS-2071. Use of isConnected() in DataXceiver is invalid. (Kihwal Lee
+ via todd)
+
+ HDFS-1981. NameNode does not saveNamespace() when editsNew is empty.
+ (Uma Maheswara Rao G via shv)
+
+ HDFS-2232. Generalize regular expressions in TestHDFSCLI.
+ (Plamen Jeliazkov via shv)
+
+ HDFS-2290. Block with corrupt replica is not getting replicated.
+ (Benoy Antony via shv)
+
+ HDFS-2452. OutOfMemoryError in DataXceiverServer takes down the DataNode
+ (Uma Maheswara Rao via cos)
+
+ HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip().
+ (Plamen Jeliazkov via shv)
+
+ HDFS-2514. Link resolution bug for intermediate symlinks with
+ relative targets. (eli)
+
+Release 0.21.1 - Unreleased
+
+ HDFS-1466. TestFcHdfsSymlink relies on /tmp/test not existing. (eli)
+
+ HDFS-874. TestHDFSFileContextMainOperations fails on weirdly
+ configured DNS hosts. (Todd Lipcon via eli)
+
+ HDFS-1507. TestAbandonBlock should abandon a block. (eli)
+
+ HDFS-1487. FSDirectory.removeBlock() should update diskspace count
+ of the block owner node (Zhong Wang via eli).
+
+ HDFS-1467. Append pipeline never succeeds with more than one replica.
+ (Todd Lipcon via eli)
+
+ HDFS-1167. New property for local conf directory in system-test-hdfs.xml
+ file. (Vinay Thota via cos)
+
+ HDFS-1503. TestSaveNamespace fails. (Todd Lipcon via cos)
+
+ HDFS-1524. Image loader should make sure to read every byte in image file.
+ (hairong)
+
+ HDFS-1523. TestLargeBlock is failing on trunk. (cos)
+
+ HDFS-1502. TestBlockRecovery triggers NPE in assert. (hairong via cos)
+
+ HDFS-1532. Exclude Findbugs warning in FSImageFormat$Saver. (Todd Lipcon
+ via cos)
+
+ HDFS-1527. SocketOutputStream.transferToFully fails for blocks >= 2GB on
+ 32 bit JVM. (Patrick Kling via cos)
+
+ HDFS-1531. Clean up stack traces due to duplicate MXBean registration.
+ (Todd Lipcon via cos)
+
+ HDFS-613. TestBalancer and TestBlockTokenWithDFS fail Balancer assert.
+ (Todd Lipcon via cos)
+
+ HDFS-1511. 98 Release Audit warnings on trunk and branch-0.22.
+ (jghoman)
+
+ HDFS-1560. dfs.data.dir permissions should default to 700.
+ (Todd Lipcon via eli)
+
+ HDFS-1550. NPE when listing a file with no location. (hairong)
+
+ HDFS-1542. Add test for HADOOP-7082, a deadlock writing Configuration to
+ HDFS. (todd)
+
+ HDFS-1504. FSImageSaver should catch all exceptions, not just IOE. (todd)
+
+ HDFS-884. DataNode throws IOException if all data directories are
+ unavailable. (Steve Loughran and shv)
+
+ HDFS-1591. HDFS part of HADOOP-6642. (Chris Douglas, Po Cheung via shv)
+
+ HDFS-900. Corrupt replicas are not processed correctly in block report (shv)
+
+ HDFS-1529. Incorrect handling of interrupts in waitForAckedSeqno can cause
+ deadlock (todd)
+
+ HDFS-1597. Batched edit log syncs can reset synctxid and throw assertions
+ (todd)
+
+ HDFS-1602. Fix HADOOP-4885 for it is doesn't work as expected. (boryas)
+
+ HDFS-1618. configure files that are generated as part of the released
+ tarball need to have executable bit set (Roman Shaposhnik via cos)
+
+ HDFS-981. test-contrib fails due to test-cactus failure (cos)
+
+ HDFS-1001. DataXceiver and BlockReader disagree on when to send/recv
+ CHECKSUM_OK. (bc Wong via eli)
+
+ HDFS-1781. Fix the path for jsvc in bin/hdfs. (John George via szetszwo)
+
+ HDFS-1782. Fix an NPE in FSNamesystem.startFileInternal(..).
+ (John George via szetszwo)
+
+ HDFS-1821. Fix username resolution in NameNode.createSymlink(..) and
+ FSDirectory.addSymlink(..). (John George via szetszwo)
+
+ HDFS-1806. TestBlockReport.blockReport_08() and _09() are timing-dependent
+ and likely to fail on fast servers. (Matt Foley via eli)
+
+ HDFS-1845. Symlink comes up as directory after namenode restart.
+ (John George via eli)
+
+ HDFS-1666. Disable failing hdfsproxy test TestAuthorizationFilter (todd)
+
+ HDFS-1823. start-dfs.sh script fails if HADOOP_HOME is not set.
+ (tomwhite via eli)
+
+Release 0.21.1 - Unreleased
+
+ HDFS-1411. Correct backup node startup command in hdfs user guide.
+ (Ching-Shen Chen via shv)
+
+ BUG FIXES
+
+ HDFS-1363. Eliminate second synchronized sections in appendFile(). (shv)
+
+ HDFS-1413. Fix broken links to HDFS Wiki. (shv)
+
+ HDFS-1420. Clover build doesn't generate per-test coverage (cos)
+
+ HDFS-1444. Test related code of build.xml is error-prone and needs to be
+ re-aligned. (cos)
+
+ HDFS-1343. Instrumented build should be concentrated in one build area (cos)
+
+ HDFS-1452. ant compile-contrib is broken (cos)
+
+ HDFS-1474. ant binary-system is broken (cos)
+
+ HDFS-1292. Allow artifacts to be published to the staging Apache Nexus
+ Maven Repository. (Giridharan Kesavan via tomwhite)
+
+ HDFS-1552. Remove java5 dependencies from build. (cos)
+
+ HDFS-1189. Quota counts missed between clear quota and set quota.
+ (John George via szetszwo)
+
+ HDFS-1665. Balancer misuses dfs.heartbeat.interval as milliseconds.
+ (szetszwo)
+
+ HDFS-1728. SecondaryNameNode.checkpointSize is in bytes but not in MB.
+ (szetszwo)
+
+ HDFS-1206. TestFiHFlush fails intermittently. (cos)
+
+ HDFS-1548. Fault-injection tests are executed multiple times if invoked
+ with run-test-hdfs-fault-inject target (cos)
+
+ HDFS-1552. Remove java5 dependencies from build. (cos)
+
+ HDFS-996. JUnit tests should never depend on anything in conf (cos)
+
+ HDFS-1612. Update HDFS design documentation for append, quota, symlink,
+ block placement and checkpoint/backup node features. (Joe Crobak
+ via szetszwo)
+
+
+ HDFS-1596. Replace fs.checkpoint.* with dfs.namenode.checkpoint.*
+ in documentations. (Harsh J Chouraria via szetszwo)
+
+ HDFS-1786. Some cli test cases expect a "null" message
+ (Uma Maheswara Rao G via todd)
+
+ HDFS-1855. TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy()
+ part 2 fails in two different ways. (Matt Foley via eli)
+
+Release 0.21.0 - 2010-08-13
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-538. Per the contract elucidated in HADOOP-6201, throw
+ FileNotFoundException from FileSystem::listStatus rather than returning
+ null. (Jakob Homan via cdouglas)
+
+ HDFS-602. DistributedFileSystem mkdirs throws FileAlreadyExistsException
+ instead of FileNotFoundException. (Boris Shkolnik via suresh)
+
+ HDFS-544. Add a "rbw" subdir to DataNode data directory. (hairong)
+
+ HDFS-576. Block report includes under-construction replicas. (shv)
+
+ HDFS-636. SafeMode counts complete blocks only. (shv)
+
+ HDFS-644. Lease recovery, concurrency support. (shv)
+
+ HDFS-570. Get last block length from a data-node when opening a file
+ being written to. (Tsz Wo (Nicholas), SZE via shv)
+
+ HDFS-657. Remove unused legacy data-node protocol methods. (shv)
+
+ HDFS-658. Block recovery for primary data-node. (shv)
+
+ HDFS-660. Remove deprecated methods from InterDatanodeProtocol. (shv)
+
+ HDFS-512. Block.equals() and compareTo() compare blocks based
+ only on block Ids, ignoring generation stamps. (shv)
+
+ HDFS-873. Configuration specifies data-node storage directories as URIs.
+ (shv)
+
+ HDFS-905. Use the new UserGroupInformation from HDFS-6299.
+ (jghoman via omalley)
+
+ HDFS-984. Persistent delegation tokens. (Jitendra Pandey via shv)
+
+ HDFS-1016. HDFS side change for HADOOP-6569. This jira changes the
+ error message on the screen when cat a directory or a
+ non-existent file. (hairong)
+
+ NEW FEATURES
+
+ HDFS-1134. Large-scale Automated Framework. (cos)
+
+ HDFS-436. Introduce AspectJ framework for HDFS code and tests.
+ (Konstantin Boudnik via szetszwo)
+
+ HDFS-447. Add LDAP lookup to hdfsproxy. (Zhiyong Zhang via cdouglas)
+
+ HDFS-459. Introduce Job History Log Analyzer. (shv)
+
+ HDFS-461. Tool to analyze file size distribution in HDFS. (shv)
+
+ HDFS-492. Add two JSON JSP pages to the Namenode for providing corrupt
+ blocks/replicas information. (Bill Zeller via szetszwo)
+
+ HDFS-578. Add support for new FileSystem method for clients to get server
+ defaults. (Kan Zhang via suresh)
+
+ HDFS-595. umask settings in configuration may now use octal or symbolic
+ instead of decimal. (Jakob Homan via suresh)
+
+ HADOOP-6234. Updated hadoop-core and test jars to propagate new option
+ dfs.umaskmode in configuration. (Jakob Homan via suresh)
+
+ HDFS-235. Add support for byte ranges in HftpFileSystem to serve
+ range of bytes from a file. (Bill Zeller via suresh)
+
+ HDFS-385. Add support for an experimental API that allows a module external
+ to HDFS to specify how HDFS blocks should be placed. (dhruba)
+
+ HADOOP-4952. Update hadoop-core and test jars to propagate new FileContext
+ file system application interface. (Sanjay Radia via suresh).
+
+ HDFS-567. Add block forensics contrib tool to print history of corrupt and
+ missing blocks from the HDFS logs.
+ (Bill Zeller, Jitendra Nath Pandey via suresh).
+
+ HDFS-610. Support o.a.h.fs.FileContext. (Sanjay Radia via szetszwo)
+
+ HDFS-536. Support hflush at DFSClient. (hairong)
+
+ HDFS-517. Introduce BlockInfoUnderConstruction to reflect block replica
+ states while writing. (shv)
+
+ HDFS-565. Introduce block committing logic during new block allocation
+ and file close. (shv)
+
+ HDFS-537. DataNode exposes a replica's meta info to BlockReceiver for the
+ support of dfs writes/hflush. It also updates a replica's bytes received,
+ bytes on disk, and bytes acked after receiving a packet. (hairong)
+
+ HDFS-585. Datanode should serve up to visible length of a replica for read
+ requests. (szetszwo)
+
+ HDFS-604. Block report processing for append. (shv)
+
+ HDFS-619. Support replica recovery initialization in datanode for the new
+ append design. (szetszwo)
+
+ HDFS-592. Allow clients to fetch a new generation stamp from NameNode for
+ pipeline recovery. (hairong)
+
+ HDFS-624. Support a new algorithm for pipeline recovery and pipeline setup
+ for append. (hairong)
+
+ HDFS-627. Support replica update in data-node.
+ (Tsz Wo (Nicholas), SZE and Hairong Kuang via shv)
+
+ HDFS-642. Support pipeline close and close error recovery. (hairong)
+
+ HDFS-631. Rename configuration keys towards API standardization and
+ backward compatibility. (Jitendra Nath Pandey via suresh)
+
+ HDFS-669. Add unit tests framework (Mockito) (cos, Eli Collins)
+
+ HDFS-731. Support new Syncable interface in HDFS. (hairong)
+
+ HDFS-702. Add HDFS implementation of AbstractFileSystem.
+ (Sanjay Radio via suresh)
+
+ HDFS-758. Add decommissioning status page to Namenode Web UI.
+ (Jitendra Nath Pandey via suresh)
+
+ HDFS-814. Add an api to get the visible length of a DFSDataInputStream.
+ (szetszwo)
+
+ HDFS-654. Add support new atomic rename functionality in HDFS for
+ supporting rename in FileContext. (suresh)
+
+ HDFS-222. Support for concatenating of files into a single file
+ without copying. (Boris Shkolnik via hairong)
+
+ HDFS-933. Adds Delegation token based authentication in the NameNode.
+ (Kan Zhang via ddas)
+
+ HDFS-935. Adds a real user component in Delegation token.
+ (Jitendra Nath Pandey via ddas)
+
+ HDFS-245. Adds a symlink implementation to HDFS. This complements the new
+ symlink feature added in HADOOP-6421 (Eli Collins via Sanjay Radia)
+
+ HDFS-1009. Support Kerberos authorization in HDFSProxy. (Srikanth
+ Sundarrajan via szetszwo)
+
+ HDFS-1091. Implement listStatus that returns an iterator of FileStatus.
+ (hairong)
+
+ IMPROVEMENTS
+
+ HDFS-381. Remove blocks from DataNode maps when corresponding file
+ is deleted. (Suresh Srinivas via rangadi)
+
+ HDFS-377. Separate codes which implement DataTransferProtocol.
+ (szetszwo)
+
+ HDFS-396. NameNode image and edits directories are specified as URIs.
+ (Luca Telloli via rangadi)
+
+ HDFS-444. Allow to change probability levels dynamically in the fault
+ injection framework. (Konstantin Boudnik via szetszwo)
+
+ HDFS-352. Documentation for saveNamespace command. (Ravi Phulari via shv)
+
+ HADOOP-6106. Updated hadoop-core and test jars from hudson trunk
+ build #12. (Giridharan Kesavan)
+
+ HDFS-204. Add a new metrics FilesInGetListingOps to the Namenode.
+ (Jitendra Nath Pandey via szetszwo)
+
+ HDFS-278. HDFS Outputstream close does not hang forever. (dhruba)
+
+ HDFS-443. Add a new metrics numExpiredHeartbeats to the Namenode.
+ (Jitendra Nath Pandey via szetszwo)
+
+ HDFS-475. Add new ant targets for fault injection jars and tests.
+ (Konstantin Boudnik via szetszwo)
+
+ HDFS-458. Create a new ant target, run-commit-test. (Jakob Homan
+ via szetszwo)
+
+ HDFS-493. Change build.xml so that the fault-injected tests are executed
+ only by the run-test-*-fault-inject targets. (Konstantin Boudnik via
+ szetszwo)
+
+ HDFS-446. Improvements to Offline Image Viewer. (Jakob Homan via shv)
+
+ HADOOP-6160. Fix releaseaudit target to run on specific directories.
+ (gkesavan)
+
+ HDFS-501. Use enum to define the constants in DataTransferProtocol.
+ (szetszwo)
+
+ HDFS-508. Factor out BlockInfo from BlocksMap. (shv)
+
+ HDFS-510. Rename DatanodeBlockInfo to be ReplicaInfo.
+ (Jakob Homan & Hairong Kuang via shv)
+
+ HDFS-500. Deprecate NameNode methods deprecated in NameNodeProtocol.
+ (Jakob Homan via shv)
+
+ HDFS-514. Change DFSClient.namenode from public to private. (Bill Zeller
+ via szetszwo)
+
+ HDFS-496. Use PureJavaCrc32 in HDFS. (Todd Lipcon via szetszwo)
+
+ HDFS-511. Remove redundant block searches in BlockManager. (shv)
+
+ HDFS-504. Update the modification time of a file when the file
+ is closed. (Chun Zhang via dhruba)
+
+ HDFS-498. Add development guide and documentation for the fault injection
+ framework. (Konstantin Boudnik via szetszwo)
+
+ HDFS-524. Further DataTransferProtocol code refactoring. (szetszwo)
+
+ HDFS-529. Use BlockInfo instead of Block to avoid redundant block searches
+ in BlockManager. (shv)
+
+ HDFS-530. Refactor TestFileAppend* to remove code duplication.
+ (Konstantin Boudnik via szetszwo)
+
+ HDFS-451. Add fault injection tests for DataTransferProtocol. (szetszwo)
+
+ HDFS-409. Add more access token tests. (Kan Zhang via szetszwo)
+
+ HDFS-546. DatanodeDescriptor iterates blocks as BlockInfo. (shv)
+
+ HDFS-457. Do not shutdown datanode if some, but not all, volumes fail.
+ (Boris Shkolnik via szetszwo)
+
+ HDFS-548. TestFsck takes nearly 10 minutes to run. (hairong)
+
+ HDFS-539. Refactor fault injeciton pipeline test util for future reuse.
+ (Konstantin Boudnik via szetszwo)
+
+ HDFS-552. Change TestFiDataTransferProtocol to junit 4 and add a few new
+ tests. (szetszwo)
+
+ HDFS-563. Simplify the codes in FSNamesystem.getBlockLocations(..).
+ (szetszwo)
+
+ HDFS-581. Introduce an iterator over blocks in the block report array.(shv)
+
+ HDFS-549. Add a new target, run-with-fault-inject-testcaseonly, which
+ allows an execution of non-FI tests in FI-enable environment. (Konstantin
+ Boudnik via szetszwo)
+
+ HDFS-173. Namenode will not block until a large directory deletion
+ completes. It allows other operations when the deletion is in progress.
+ (suresh)
+
+ HDFS-551. Create new functional test for a block report. (Konstantin
+ Boudnik via hairong)
+
+ HDFS-288. Redundant computation in hashCode() implementation.
+ (szetszwo via tomwhite)
+
+ HDFS-412. Hadoop JMX usage makes Nagios monitoring impossible.
+ (Brian Bockelman via tomwhite)
+
+ HDFS-472. Update hdfsproxy documentation. Adds a setup guide and design
+ document. (Zhiyong Zhang via cdouglas)
+
+ HDFS-617. Support non-recursive create(). (Kan Zhang via szetszwo)
+
+ HDFS-618. Support non-recursive mkdir(). (Kan Zhang via szetszwo)
+
+ HDFS-574. Split the documentation between the subprojects.
+ (Corinne Chandel via omalley)
+
+ HDFS-598. Eclipse launch task for HDFS. (Eli Collins via tomwhite)
+
+ HDFS-641. Move all of the components that depend on map/reduce to
+ map/reduce. (omalley)
+
+ HDFS-509. Redesign DataNode volumeMap to include all types of Replicas.
+ (hairong)
+
+ HDFS-562. Add a test for NameNode.getBlockLocations(..) to check read from
+ un-closed file. (szetszwo)
+
+ HDFS-543. Break FSDatasetInterface#writToBlock() into writeToRemporary,
+ writeToRBW, ad append. (hairong)
+
+ HDFS-603. Add a new interface, Replica, which is going to replace the use
+ of Block in datanode. (szetszwo)
+
+ HDFS-589. Change block write protocol to support pipeline recovery.
+ (hairong)
+
+ HDFS-652. Replace BlockInfo.isUnderConstruction() with isComplete() (shv)
+
+ HDFS-648. Change some methods in AppendTestUtil to public. (Konstantin
+ Boudnik via szetszwo)
+
+ HDFS-662. Unnecessary info message from DFSClient. (hairong)
+
+ HDFS-518. Create new tests for Append's hflush. (Konstantin Boudnik
+ via szetszwo)
+
+ HDFS-688. Add configuration resources to DFSAdmin. (shv)
+
+ HDFS-29. Validate the consistency of the lengths of replica and its file
+ in replica recovery. (szetszwo)
+
+ HDFS-680. Add new access method to a copy of a block's replica. (shv)
+
+ HDFS-704. Unify build property names to facilitate cross-projects
+ modifications (cos)
+
+ HDFS-705. Create an adapter to access some of package-private methods of
+ DataNode from tests (cos)
+
+ HDFS-710. Add actions with constraints to the pipeline fault injection
+ tests and change SleepAction to support uniform random sleeping over an
+ interval. (szetszwo)
+
+ HDFS-713. Need to properly check the type of the test class from an aspect
+ (cos)
+
+ HDFS-716. Define a pointcut for pipeline close and add a few fault
+ injection tests to simulate out of memory problem. (szetszwo)
+
+ HDFS-719. Add 6 fault injection tests for pipeline close to simulate slow
+ datanodes and disk errors. (szetszwo)
+
+ HDFS-616. Create functional tests for new design of the block report. (cos)
+
+ HDFS-584. Fail the fault-inject build if any advices are mis-bound. (cos)
+
+ HDFS-730. Add 4 fault injection tests to simulate non-responsive datanode
+ and out-of-memory problem for pipeline close ack. (szetszwo)
+
+ HDFS-728. Create a comprehensive functional test for append. (hairong)
+
+ HDFS-736. commitBlockSynchronization() updates block GS and length
+ in-place. (shv)
+
+ HADOOP-5107. Use Maven ant tasks to publish the subproject jars.
+ (Giridharan Kesavan via omalley)
+
+ HDFS-521. Create new tests for pipeline (cos)
+
+ HDFS-764. Places the Block Access token implementation in hdfs project.
+ (Kan Zhang via ddas)
+
+ HDFS-787. Upgrade some libraries to be consistent with common and
+ mapreduce. (omalley)
+
+ HDFS-519. Create new tests for lease recovery (cos)
+
+ HDFS-804. New unit tests for concurrent lease recovery (cos)
+
+ HDFS-813. Enable the append test in TestReadWhileWriting. (szetszwo)
+
+ HDFS-145. Cleanup inconsistent block length handling code in
+ FSNameSystem#addStoredBlock. (hairong)
+
+ HDFS-127. Reset failure count in DFSClient for each block acquiring
+ operation. (Igor Bolotin via szetszwo)
+
+ HDFS-520. Create new tests for block recovery. (hairong)
+
+ HDFS-1067. Create block recovery tests that handle errors. (hairong)
+
+ HDFS-1107. Turn on append by default. (shv)
+
+ HDFS-968. Use StringBuilder instead of StringBuffer for better
+ performance. (Kay Kay via suresh)
+
+ HDFS-703. Replace current fault injection implementation with one
+ from (cos)
+
+ HDFS-754. Reduce ivy console output to observable level (cos)
+
+ HDFS-832. HDFS side of HADOOP-6222. (cos)
+
+ HDFS-840. Change tests to use FileContext test helper introduced in
+ HADOOP-6394. (Jitendra Nath Pandey via suresh)
+
+ HDFS-685. Use the user-to-groups mapping service in the NameNode.
+ (boryas, acmurthy)
+
+ HDFS-755. Read multiple checksum chunks at once in DFSInputStream.
+ (Todd Lipcon via tomwhite)
+
+ HDFS-786. Implement getContentSummary in HftpFileSystem.
+ (Tsz Wo (Nicholas), SZE via cdouglas)
+
+ HDFS-587. Add support for specifying queue name in mapreduce tests.
+ (Erik Steffl via suresh)
+
+ HDFS-902 Move contrib/raid to MapReduce. (Eli Collins via omalley)
+
+ HDFS-800. The last block of a file under construction may change to the
+ COMPLETE state in response to getAdditionalBlock or completeFileInternal.
+ (hairong)
+
+ HDFS-899. Delegation Token Implementation
+ and corresponding changes in Namenode and DFS Api to issue,
+ renew and cancel delegation tokens. (jnp via boryas)
+
+ HDFS-844. Log the filename when file locking fails. (tomwhite)
+
+ HDFS-914. Refactor DFSOutputStream and DFSInputStream out of DFSClient.
+ (Todd Lipcon via tomwhite)
+
+ HDFS-949. Move DelegationToken into Common so that it can be used by
+ MapReduce. (omalley)
+
+ HDFS-930. Better error message for DATA_TRANSFER_VERSION mismatched.
+ (Kay Kay via szetszwo)
+
+ HDFS-986. Delegation token renewing and cancelling should provide
+ meaningful exceptions when there are failures instead of returning
+ false. (omalley)
+
+ HADOOP-6579. Upgrade the commons-codec library to 1.4. (omalley)
+
+ HDFS-991. Allow authentication to the web ui via a delegation token.
+ (omalley)
+
+ HDFS-994. Allow fetching of delegation token from NameNode for hftp.
+ (Jakob Homan via acmurthy)
+
+ HDFS-998. Quote blocks streamed through jsps. (cdouglas)
+
+ HDFS-729. NameNode API to list files that have missing blocks.
+ (Rodrigo Schmidt via dhruba)
+
+ HDFS-850. The WebUI display more details about namenode memory usage.
+ (Dmytro Molkov via dhruba)
+
+ HDFS-826. The DFSOutputStream has a API that returns the number of
+ active datanode(s) in the current pipeline. (dhruba)
+
+ HDFS-985. HDFS should issue multiple RPCs for listing a large
+ directory. (hairong)
+
+ HDFS-1043. NNThroughputBenchmark modifications to support benchmarking of
+ server-side user group resolution. (shv)
+
+ HDFS-892. Optionally use Avro reflection for Namenode RPC. This
+ is not a complete implementation yet, but rather a starting point.
+ (cutting)
+
+ HDFS-854. Datanode should scan devices in parallel to generate
+ block report. (Dmytro Molkov via jhoman)
+
+ HDFS-1032. fsck has an option to list corrupt files.
+ (Andre Oriai via dhruba)
+
+ HDFS-1024. SecondaryNameNode verifies size of fsimage and edits file.
+ (Dmytro Molkov via dhruba)
+
+ HDFS-1011. hdfsproxy: Improve log messages by restoring the previous
+ thread name. (Srikanth Sundarrajan via szetszwo)
+
+ HDFS-997. Allow datanode storage directory permissions to be configurable.
+ (Luke Lu via cdouglas)
+
+ HDFS-1012. hdfsproxy: Support for fully qualified HDFS path in addition to
+ simple unqualified path. (Srikanth Sundarrajan via szetszwo)
+
+ HDFS-933. Namenode should issue a delegation token only for kerberos
+ authenticated clients.(jnp via boryas)
+
+ HDFS-1087. Modify audit log to use a StringBuilder rather than a Formatter.
+ (cdouglas)
+
+ HDFS-1083. Update TestHDFSCLI not to expect exception class name
+ in error messages. (suresh)
+
+ HDFS-1099. Add test for umask backward compatibility. (suresh)
+
+ HDFS-1092. Use logging rather than System.err in MiniDFSCluster.
+ (Kay Kay via jghoman)
+
+ HDFS-1047. Install/deploy source jars to Maven repo.
+ (Patrick Angeles via jghoman)
+
+ HDFS-666. Unit test for FsShell -text. (cdouglas via jghoman)
+
+ HDFS-1054. Remove unnecessary sleep after failure in nextBlockOutputStream.
+ (Todd Lipcon via jghoman)
+
+ HDFS-921. Convert TestDFSClientRetries::testNotYetReplicatedErrors
+ to Mockito. (jghoman)
+
+ HDFS-1100. Override unwrapException in TestFcHdfsSymlink to test
+ symlink API conformance. (Eli Collins via suresh).
+
+ HDFS-1089. Remove uses of FileContext#isFile, isDirectory, and exists.
+ (Eli Collins via hairong)
+
+ HDFS-1028. Efficient splitting of path components reduces the time
+ to load in fsimage by 20%. (Dmytro Molkov via dhruba)
+
+ HDFS-1109. HFTP supports filenames that contains the character "+".
+ (Dmytro Molkov via dhruba)
+
+ HDFS-853. The HDFS webUI displays the balanced-ness of the cluster.
+ (Dmytro Molkov via dhruba)
+
+ HDFS-1126. Change HDFS to depend on Hadoop 'common' artifacts instead
+ of 'core'. (tomwhite)
+
+ HDFS-995. Replace usage of FileStatus#isDir(). (Eli Collins via
+ tomwhite)
+
+ HDFS-1161. Make DN minimum valid volumes configurable.
+ (Eli Collins via tomwhite)
+
+ HDFS-1181. Move configuration and script files post split. (tomwhite)
+
+ HDFS-1170. Add more assertions to TestLargeDirectoryDelete.
+ (Steve Loughran via tomwhite)
+
+ HDFS-1199. Extract a subset of tests for smoke (DOA) validation. (cos)
+
+ HDFS-1174. New properties for suspend and resume process. (Vinay Thota via
+ cos)
+
+ HDFS-1277. [Herriot] New property for multi user list. (Vinay Thota via
+ cos)
+
+ HDFS-806. Add new unit tests to the 10-mins 'run-commit-test' target (cos)
+
+ OPTIMIZATIONS
+
+ HDFS-946. NameNode should not return full path name when lisitng a
+ diretory or getting the status of a file. (hairong)
+
+ BUG FIXES
+
+ HDFS-76. Better error message to users when commands fail because of
+ lack of quota. Allow quota to be set even if the limit is lower than
+ current consumption. (Boris Shkolnik via rangadi)
+
+ HADOOP-4687. HDFS is split from Hadoop Core. It is a subproject under
+ Hadoop (Owen O'Malley)
+
+ HADOOP-6096. Fix Eclipse project and classpath files following project
+ split. (tomwhite)
+
+ HDFS-195. Handle expired tokens when write pipeline is reestablished.
+ (Kan Zhang via rangadi)
+
+ HDFS-181. Validate src path in FSNamesystem.getFileInfo(..). (Todd
+ Lipcon via szetszwo)
+
+ HDFS-441. Remove TestFTPFileSystem. (szetszwo)
+
+ HDFS-440. Fix javadoc broken links in DFSClient. (szetszwo)
+
+ HDFS-480. Fix a typo in the jar name in build.xml.
+ (Konstantin Shvachko via gkesavan)
+
+ HDFS-438. Check for NULL before invoking GenericArgumentParser in
+ DataNode. (Raghu Angadi)
+
+ HDFS-415. BlockReceiver hangs in case of certain runtime exceptions.
+ (Konstantin Boudnik via rangadi)
+
+ HDFS-462. loadFSImage should close edits file. (Jakob Homan via shv)
+
+ HDFS-489. Update TestHDFSCLI for the -skipTrash option in rm. (Jakob Homan
+ via szetszwo)
+
+ HDFS-445. pread() does not pick up changes to block locations.
+ (Kan Zhang via rangadi)
+
+ HDFS-463. CreateEditLog utility broken after HDFS-396 (URI for
+ FSImage). (Suresh Srinivas via rangadi)
+
+ HDFS-484. Fix bin-package and package target to package jar files.
+ (gkesavan)
+
+ HDFS-490. Eliminate the deprecated warnings introduced by H-5438.
+ (He Yongqiang via szetszwo)
+
+ HDFS-119. Fix a bug in logSync(), which causes NameNode block forever.
+ (Suresh Srinivas via shv)
+
+ HDFS-534. Include avro in ivy. (szetszwo)
+
+ HDFS-532. Allow applications to know that a read request failed
+ because block is missing. (dhruba)
+
+ HDFS-561. Fix write pipeline READ_TIMEOUT in DataTransferProtocol.
+ (Kan Zhang via szetszwo)
+
+ HDFS-553. BlockSender reports wrong failed position in ChecksumException.
+ (hairong)
+
+ HDFS-568. Set mapred.job.tracker.retire.jobs to false in
+ src/test/mapred-site.xml for mapreduce tests to run. (Amareshwari
+ Sriramadasu via szetszwo)
+
+ HDFS-15. All replicas end up on 1 rack. (Jitendra Nath Pandey via hairong)
+
+ HDFS-586. TestBlocksWithNotEnoughRacks sometimes fails.
+ (Jitendra Nath Pandey via hairong)
+
+ HADOOP-6243. Fixed a NullPointerException in handling deprecated keys.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HDFS-605. Do not run fault injection tests in the run-test-hdfs-with-mr
+ target. (Konstantin Boudnik via szetszwo)
+
+ HDFS-606. Fix ConcurrentModificationException in invalidateCorruptReplicas()
+ (shv)
+
+ HDFS-601. TestBlockReport obtains data directories directly from
+ MiniHDFSCluster. (Konstantin Boudnik via shv)
+
+ HDFS-614. TestDatanodeBlockScanner obtains data directories directly from
+ MiniHDFSCluster. (shv)
+
+ HDFS-612. Remove the use of org.mortbay.log.Log in FSDataset. (szetszwo)
+
+ HDFS-622. checkMinReplication should count live nodes only. (shv)
+
+ HDFS-629. Remove ReplicationTargetChooser.java along with fixing
+ import warnings generated by Eclipse. (dhruba)
+
+ HDFS-637. DataNode sends a Success ack when block write fails. (hairong)
+
+ HDFS-640. Fixed TestHDFSFileContextMainOperations.java build failure. (suresh)
+
+ HDFS-547. TestHDFSFileSystemContract#testOutputStreamClosedTwice
+ sometimes fails with CloseByInterruptException. (hairong)
+
+ HDFS-588. Fix TestFiDataTransferProtocol and TestAppend2 failures. (shv)
+
+ HDFS-550. DataNode restarts may introduce corrupt/duplicated/lost replicas
+ when handling detached replicas. (hairong)
+
+ HDFS-659. If the the last block is not complete, update its length with
+ one of its replica's length stored in datanode. (szetszwo)
+
+ HDFS-649. Check null pointers for DataTransferTest. (Konstantin Boudnik
+ via szetszwo)
+
+ HDFS-661. DataNode upgrade fails on non-existant current directory.
+ (hairong)
+
+ HDFS-597. Mofication introduced by HDFS-537 breakes an advice binding in
+ FSDatasetAspects. (Konstantin Boudnik via szetszwo)
+
+ HDFS-665. TestFileAppend2 sometimes hangs. (hairong)
+
+ HDFS-676. Fix NPE in FSDataset.updateReplicaUnderRecovery() (shv)
+
+ HDFS-673. BlockReceiver#PacketResponder should not remove a packet from
+ the ack queue before its ack is sent. (hairong)
+
+ HDFS-682. Fix bugs in TestBlockUnderConstruction. (szetszwo)
+
+ HDFS-668. TestFileAppend3#TC7 sometimes hangs. (hairong)
+
+ HDFS-679. Appending to a partial chunk incorrectly assumes the
+ first packet fills up the partial chunk. (hairong)
+
+ HDFS-722. Fix callCreateBlockWriteStream pointcut in FSDatasetAspects.
+ (szetszwo)
+
+ HDFS-690. TestAppend2#testComplexAppend failed on "Too many open files".
+ (hairong)
+
+ HDFS-725. Support the build error fix for HADOOP-6327. (Sanjay Radia via
+ szetszwo)
+
+ HDFS-625. Fix NullPointerException thrown from ListPathServlet. (suresh)
+
+ HDFS-735. TestReadWhileWriting has wrong line termination symbols (cos)
+
+ HDFS-691. Fix an overflow error in DFSClient.DFSInputStream.available().
+ (szetszwo)
+
+ HDFS-733. TestBlockReport fails intermittently. (cos)
+
+ HDFS-774. Intermittent race condition in TestFiPipelines (cos)
+
+ HDFS-741. TestHFlush test doesn't seek() past previously written part of
+ the file (cos, szetszwo)
+
+ HDFS-706. Intermittent failures in TestFiHFlush (cos)
+
+ HDFS-646. Fix test-patch failure by adding test-contrib ant target.
+ (gkesavan)
+
+ HDFS-791. Build is broken after HDFS-787 patch has been applied (cos)
+
+ HDFS-792. TestHDFSCLI is failing. (Todd Lipcon via cos)
+
+ HDFS-781. Namenode metrics PendingDeletionBlocks is not decremented.
+ (Suresh)
+
+ HDFS-192. Fix TestBackupNode failures. (shv)
+
+ HDFS-797. TestHDFSCLI much slower after HDFS-265 merge. (Todd Lipcon via cos)
+
+ HDFS-824. Stop lease checker in TestReadWhileWriting. (szetszwo)
+
+ HDFS-823. CheckPointer should use addInternalServlet for image-fetching
+ servlet (jghoman)
+
+ HDFS-456. Fix URI generation for windows file paths. (shv)
+
+ HDFS-812. FSNamesystem#internalReleaseLease throws NullPointerException on
+ a single-block file's lease recovery. (cos)
+
+ HDFS-724. Pipeline hangs if one of the block receiver is not responsive.
+ (hairong)
+
+ HDFS-564. Adding pipeline tests 17-35. (hairong)
+
+ HDFS-849. TestFiDataTransferProtocol2#pipeline_Fi_18 sometimes fails.
+ (hairong)
+
+ HDFS-762. Balancer causes Null Pointer Exception.
+ (Cristian Ivascu via dhruba)
+
+ HDFS-868. Fix link to Hadoop Upgrade Wiki. (Chris A. Mattmann via shv)
+
+ HDFS-880. TestNNLeaseRecovery fails on windows (cos, shv)
+
+ HDFS-699. Primary datanode should compare replicas' on disk lengths.
+ (hairong)
+
+ HDFS-897. Fix a bug related to generation stamp comparison in
+ ReplicasMap. (suresh)
+
+ HDFS-793. Data node should receive the whole packet ack message before it
+ constructs and sends its own ack message for the packet. (hairong)
+
+ HDFS-101. DFS write pipeline: DFSClient sometimes does not detect second
+ datanode failure. (hairong)
+
+ HDFS-822. Appends to already-finalized blocks can rename across volumes.
+ (hairong)
+
+ HDFS-1046. Fix Tomcat version in hdfsproxy/build.xml. (Srikanth
+ Sundarrajan via szetszwo)
+
+ HDFS-1072. Fix TestReadWhileWriting failure. (Erik Steffl via shv)
+
+ HDFS-913. Rename fault injection test TestRename.java to TestFiRename.java
+ to include it in tests run by ant target run-test-hdfs-fault-inject.
+ (suresh)
+
+ HDFS-695. RaidNode should read in configuration from hdfs-site.xml.
+ (dhruba)
+
+ HDFS-726. Eclipse .classpath template has outdated jar files and is
+ missing some new ones. (cos)
+
+ HDFS-750. Fix build failure due to TestRename. (suresh)
+
+ HDFS-712. Move libhdfs from mapreduce subproject to hdfs subproject.
+ (Eli Collins via dhruba)
+
+ HDFS-757. Enable Unit test for HDFS Raid. (dhruba)
+
+ HDFS-611. Prevent DataNode heartbeat times from increasing even when
+ the DataNode has many blocks to delete. (Zheng Shao via dhruba)
+
+ HDFS-751. Fix TestCrcCorruption to pick up the correct datablocks to
+ corrupt. (dhruba)
+
+ HDFS-763. Fix slightly misleading report from DataBlockScanner
+ about corrupted scans. (dhruba)
+
+ HDFS-727. bug setting block size hdfsOpenFile (Eli Collins via cos)
+
+ HDFS-756. libhdfs unit tests do not run. (Eli Collins via cos)
+
+ HDFS-783. libhdfs tests brakes code coverage runs with Clover (cos)
+
+ HDFS-785. Add Apache license to several namenode unit tests.
+ (Ravi Phulari via jghoman)
+
+ HDFS-802. Update Eclipse configuration to match changes to Ivy
+ configuration (Edwin Chan via cos)
+
+ HDFS-423. Unbreak FUSE build and fuse_dfs_wrapper.sh (Eli Collins via cos)
+
+ HDFS-825. Build fails to pull latest hadoop-core-* artifacts (cos)
+
+ HDFS-94. The Heap Size printed in the NameNode WebUI is accurate.
+ (Dmytro Molkov via dhruba)
+
+ HDFS-767. An improved retry policy when the DFSClient is unable to fetch a
+ block from the datanode. (Ning Zhang via dhruba)
+
+ HDFS-775. FSDataset calls getCapacity() twice. (stevel)
+
+ HDFS-885. Datanode toString() NPEs on null dnRegistration. (stevel)
+
+ HDFS-877. Client-driven block verification not functioning. (Todd
+ Lipcon via hairong)
+
+ HDFS-630. In DFSOutputStream.nextBlockOutputStream(), the client can
+ exclude specific datanodes when locating the next block.
+ (Cosmin Lehene via Stack)
+
+ HDFS-922. Remove unnecessary semicolon added by HDFS-877 that causes
+ problems for Eclipse compilation. (jghoman)
+
+ HDFS-927 DFSInputStream retries too many times for new block locations
+ (Todd Lipcon via Stack)
+
+ HDFS-938. Replace calls to UGI.getUserName() with UGI.getShortUserName()
+ (jghoman)
+
+ HDFS-894. DatanodeID.ipcPort is not updated when existing node
+ re-registers. (Todd Lipcon via tomwhite)
+
+ HDFS-965. Split TestDelegationToken in to two parts and fix configuration
+ to allow proxy users in the test. (Jitendra Pandey via omalley)
+
+ HDFS-999. Secondary namenode should login using kerberos if security is
+ configured (boryas)
+
+ HDFS-856. Hardcoded replication level for new files in fuse-dfs.
+ (Brian Bockelman via tomwhite)
+
+ HDFS-857. Incorrect type for fuse-dfs capacity can cause "df" to return
+ negative values on 32-bit machines. (Brian Bockelman via tomwhite)
+
+ HDFS-858. Incorrect return codes for fuse-dfs. (Brian Bockelman via
+ tomwhite)
+
+ HDFS-859. fuse-dfs utime behavior causes issues with tar.
+ (Brian Bockelman via tomwhite)
+
+ HDFS-861. fuse-dfs does not support O_RDWR. (Brian Bockelman via tomwhite)
+
+ HDFS-961. dfs_readdir incorrectly parses paths. (Eli Collins via tomwhite)
+
+ HDFS-1015. Fix intermittent failure in TestSecurityTokenEditLog.
+ (Jitendra Nath Pandey via suresh)
+
+ HDFS-939. libhdfs test is broken. (Eli Collins via tomwhite)
+
+ HDFS-1074. hdfsproxy: Fix bugs in TestProxyUtil. (Srikanth Sundarrajan
+ via szetszwo)
+
+ HDFS-481. hdfsproxy: Bug Fixes + HdfsProxy to use proxy user to
+ impresonate the real user. (Srikanth Sundarrajan via szetszwo)
+
+ HDFS-482. Move HsftpFileSystem's ssl.client.do.not.authenticate.server
+ configuration setting to ssl-client.xml. (Srikanth Sundarrajan via
+ szetszwo)
+
+ HDFS-1010. hdfsproxy: Retrieve groups from UnixUserGroupInformation
+ instead of LdapEntry. (Srikanth Sundarrajan via szetszwo)
+
+ HDFS-466. hdfs_write infinite loop when dfs fails and cannot write
+ files > 2 GB. (Pete Wyckoff via tomwhite)
+
+ HDFS-651. HDFS Docs - fix listing of docs in the doc menu.
+ (Corinne Chandel via tomwhite)
+
+ HDFS-1014. Error in reading delegation tokens from edit logs.
+ (Jitendra Nath Pandey via jhoman)
+
+ HDFS-1088. Prevent renaming a symbolik link to its target.
+ (Eli Collins via suresh)
+
+ HDFS-966. NameNode does not recovers lease when it is in safemode.
+ (dhruba)
+
+ HDFS-833. Datanode shutdown should log problems with Storage.unlockAll()
+ (Steve Loughran via dhruba)
+
+ HDFS-1101. TestDiskError.testLocalDirs() fails. (cdouglas via jghoman)
+
+ HDFS-1031. Enhance the webUi to list a few of the corrupted files in HDFS.
+ (Andre Orian via dhruba)
+
+ HDFS-1078. Create static and dynamic versions of libhdfs.
+ (Sam Rash via dhruba)
+
+ HDFS-1104. Fsck triggers full GC on NameNode. (hairong)
+
+ HDFS-1141. Closing a file is successful only if the client still has a
+ valid lease. (Todd Lipcon via dhruba)
+
+ HDFS-1138. Prevent erroneous updation of modification time of a directory
+ when fsimage loads. (Dmytro Molkov via dhruba)
+
+ HDFS-1000. Updates libhdfs to the new API for UGI (ddas)
+
+ HDFS-609. Create a file with the append flag does not work in HDFS.
+ (tomwhite)
+
+ HDFS-1255. Fix failing test-libhdfs.sh test. (tomwhite)
+
+ HDFS-1256. libhdfs is missing from the tarball. (tomwhite)
+
+ HDFS-1057. Concurrent readers hit ChecksumExceptions if following a
+ writer to very end of file. (sam rash via hairong)
+
+ HDFS-1212. Harmonize HDFS JAR library versions with Common. (tomwhite)
+
+ HDFS-1159. clean-cache target removes wrong ivy cache (cos)
+
+ HDFS-1193. -mvn-system-deploy target is broken which inturn fails the
+ mvn-deploy task leading to unstable mapreduce build (Giridharan
+ Kesavan via cos)
+
+ HDFS-1299. 'compile-fault-inject' never should be called directly. (cos)
+
+ HDFS-1311. Running tests with 'testcase' cause triple execution of the
+ same test case (Cos)
+
+ HDFS-1267. fuse-dfs does not compile. (Devaraj Das via tomwhite)
+
+ HDFS-1598. Directory listing on hftp:// does not show .*.crc files.
+ (szetszwo)
+
+ HDFS-1750. ListPathsServlet should not use HdfsFileStatus.getLocalName()
+ to get file name since it may return an empty string. (szetszwo)
+
+Release 0.20.3 - Unreleased
+
+ IMPROVEMENTS
+
+ BUG FIXES
+
+ HDFS-1041. DFSClient.getFileChecksum(..) should retry if connection to
+ the first datanode fails. (szetszwo)
+
+ HDFS-909. Wait until edits syncing is finishes before purging edits.
+ (Todd Lipcon via shv)
+
+ HDFS-1258. Clearing namespace quota on "/" corrupts fs image.
+ (Aaron T. Myers via szetszwo)
+
+ HDFS-1406. TestCLI fails on Ubuntu with default /etc/hosts. (cos)
+
+Release 0.20.203.0 - 2011-5-11
+
+ IMPROVEMENTS
+
+ HADOOP-7259. Contrib modules should include the build.properties from
+ the enclosing hadoop directory. (omalley)
+
+ BUG FIXES
+
+ HDFS-132. Fix namenode to not report files deleted metrics for deletions
+ done while replaying edits during startup. (suresh & shv)
+
+ HDFS-955. New implementation of saveNamespace() to avoid loss of edits
+ when name-node fails during saving. (shv)
+
+Release 0.20.2 - 2009-09-01
+
+ IMPROVEMENTS
+
+ HDFS-737. Add full path name of the file to the block information and
+ summary of total number of files, blocks, live and deadnodes to
+ metasave output. (Jitendra Nath Pandey via suresh)
+
+ HDFS-919. Create test to validate the BlocksVerified metric (Gary Murry
+ via cos)
+
+ HDFS-907. Add tests for getBlockLocations and totalLoad metrics.
+ (Ravi Phulari via cos)
+
+ BUG FIXES
+
+ HDFS-686. NullPointerException is thrown while merging edit log and image.
+ (hairong)
+
+ HDFS-677. Rename failure when both source and destination quota exceeds
+ results in deletion of source. (suresh)
+
+ HDFS-709. Fix TestDFSShell failure due to rename bug introduced by
+ HDFS-677. (suresh)
+
+ HDFS-579. Fix DfsTask to follow the semantics of 0.19, regarding non-zero
+ return values as failures. (Christian Kunz via cdouglas)
+
+ HDFS-723. Fix deadlock in DFSClient#DFSOutputStream. (hairong)
+
+ HDFS-596. Fix memory leak in hdfsFreeFileInfo() for libhdfs.
+ (Zhang Bingjun via dhruba)
+
+ HDFS-185. Disallow chown, chgrp, chmod, setQuota, and setSpaceQuota when
+ name-node is in safemode. (Ravi Phulari via shv)
+
+ HDFS-187. Initialize secondary namenode http address in TestStartup.
+ (Todd Lipcon via szetszwo)
+
+ HDFS-464. Fix memory leaks in libhdfs. (Christian Kunz via suresh)
+
+ HDFS-1377. Quota bug for partial blocks allows quotas to be violated. (eli)
+
+Release 0.20.1 - 2009-09-01
+
+ IMPROVEMENTS
+
+ HDFS-438. Improve help message for space quota command. (Raghu Angadi)
+
+ BUG FIXES
+
+ HDFS-167. Fix a bug in DFSClient that caused infinite retries on write.
+ (Bill Zeller via szetszwo)
+
+ HDFS-527. Remove/deprecate unnecessary DFSClient constructors. (szetszwo)
+
+ HDFS-525. The SimpleDateFormat object in ListPathsServlet is not thread
+ safe. (Suresh Srinivas and cdouglas)
+
+ HDFS-761. Fix failure to process rename operation from edits log due to
+ quota verification. (suresh)