Big data hadoop practice test2

Thank you very much for taking test

Test your caliber

1. In a MapReduce job, you want each of your input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies?
Increase the parameter that controls minimum split size in the job configuration.	Write a custom MapRunner that iterates over all key-value pairs in the entire file.
Set the number of mappers equal to the number of input files you want to process.	Write a custom FileInputFormat and override the method isSplitable to always return false.

2. Which process describes the lifecycle of a Mapper?
The JobTracker calls the TaskTracker’s configure () method, then its map () method and finally its close () method.	The TaskTracker spawns a new Mapper to process all records in a single input split.
The TaskTracker spawns a new Mapper to process each key-value pair.	The JobTracker spawns a new Mapper to process all records in a single file.

3. Determine which best describes when the reduce method is first called in a MapReduce job?
Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The programmer can configure in the job what percentage of the intermediate data should arrive before the reduce method begins.	Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The reduce method is called only after all intermediate data has been copied and sorted.
Reduce methods and map methods all start at the beginning of a job, in order to provide optimal performance for map-only or reduce-only jobs.	Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The reduce method is called as soon as the intermediate key-value pairs start to arrive.

4. You have written a Mapper which invokes the following five calls to the OutputColletor.collect method: output.collect (new Text (“Apple”), new Text (“Red”) ) ; output.collect (new Text (“Banana”), new Text (“Yellow”) ) ; output.collect (new Text (“Apple”), new Text (“Yellow”) ) ; output.collect (new Text (“Cherry”), new Text (“Red”) ) ; output.collect (new Text (“Apple”), new Text (“Green”) ) ; How many times will the Reducer’s reduce method be invoked?
6	3
1	0

5. To process input key-value pairs, your mapper needs to lead a 512 MB data file in memory. What is the best way to accomplish this?
Serialize the data file, insert in it the JobConf object, and read the data into memory in the configure method of the mapper.	Place the data file in the DistributedCache and read the data into memory in the map method of the mapper.
Place the data file in the DataCache and read the data into memory in the configure method of the mapper.	Place the data file in the DistributedCache and read the data into memory in the configure method of the mapper.

6. In a MapReduce job, the reducer receives all values associated with same key. Which statement best describes the ordering of these values?
The values are in sorted order.	The values are arbitrarily ordered, and the ordering may vary from run to run of the same MapReduce job.
The values are arbitrary ordered, but multiple runs of the same MapReduce job will always have the same ordering.	Since the values come from mapper outputs, the reducers will receive contiguous sections of sorted values.

7. You need to create a job that does frequency analysis on input data. You will do this by writing a Mapper that uses TextInputFormat and splits each value (a line of text from an input file) into individual characters. For each one of these characters, you will emit the character as a key and an InputWritable as the value. As this will produce proportionally more intermediate data than input data, which two resources should you expect to be bottlenecks?
Processor and network I/O	Disk I/O and network I/O
Processor and RAM	Processor and disk I/O

8. You want to count the number of occurrences for each unique word in the supplied input data. You’ve decided to implement this by having your mapper tokenize each word and emit a literal value 1, and then have your reducer increment a counter for each literal 1 it receives. After successful implementing this, it occurs to you that you could optimize this by specifying a combiner. Will you be able to reuse your existing Reduces as your combiner in this case and why or why not?
Yes, because the sum operation is both associative and commutative and the input and output types to the reduce method match.	No, because the sum operation in the reducer is incompatible with the operation of a Combiner.
No, because the Reducer and Combiner are separate interfaces.	No, because the Combiner is incompatible with a mapper which doesn’t use the same data type for both the key and value.

9. Your client application submits a MapReduce job to your Hadoop cluster. Identify the Hadoop daemon on which the Hadoop framework will look for an available slot schedule a MapReduce operation.
TaskTracker	NameNode
DataNode	JobTracker

10. Which project gives you a distributed, Scalable, data store that allows you random, realtime read/write access to hundreds of terabytes of data?
HBase	Hue
Pig	Hive

11. What is a SequenceFile?
A Sequence Filecontains a binary encoding of an arbitrary number of homo geneous writable objects.	A Sequence Filecontains a binary encoding of an arbitrary number of hetero geneous writable objects.
A Sequence Filecontains a binary encoding of an arbitrary number of Writable Comparable objects, in sorted order.	A Sequence Filecontains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be same type.

12. In a MapReduce job, you want each of you input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies?
Increase the parameter that controls minimum split size in the job configuration.	Write a custom MapRunner that iterates over all key-value pairs in the entire file.
Set the number of mappers equal to the number of input files you want to process.	Write a custom FileInputFormat and override the method isSplittable to always return false.

13. Which of the following best describes the workings of TextInputFormat?
Input file splits may cross line breaks. A line thatcrosses tile splits is ignored.	The input file is split exactly at the line breaks, so each Record Reader will read a series of complete lines.
Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReaders of both splits containing the brokenline.	Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReader of the split that contains the end of the broken line.

14. Which of the following statements most accurately describes the relationship between MapReduce and Pig?
Pig provides additional capabilities that allow certain types of data manipulation not possible with MapReduce.	Pig provides no additional capabilities to MapReduce. Pig programs are executed as MapReduce jobs via the Pig interpreter.
Pig programs rely on MapReduce but are extensible, allowing developers to do specialpurpose processing not provided by MapReduce.	Pig provides the additional capability of allowing you to control the flow of multiple MapReduce jobs.

15. Will settings using Java API overwrite values in configuration files?
No. The configuration settings in the configuration file takes precedence	Yes. The configuration settings using Java API take precedence
It depends when the developer reads the configuration file. If it is read first then no.	Only global configuration settings are captured in configuration files on namenode. There are only a very few job parameters that can be set using Java API.

16. What is AVRO?
Avro is a java serialization library	Avro is a java compression library
Avro is a java library that create splittable files	None of these answers are correct

17. Can you run Map - Reduce jobs directly on Avro data?
Yes, Avro was specifically designed for data processing via Map-Reduce	Yes, but additional extensive coding is required
No, Avro was specifically designed for data storage only	Avro specifies metadata that allows easier data access. This data cannot be used as part of mapreduce execution, rather input specification only.

18. What is distributed cache?
The distributed cache is special component on namenode that will cache frequently used data for faster client response. It is used during reduce step.	The distributed cache is special component on datanode that will cache frequently used data for faster client response. It is used during map step.
The distributed cache is a component that caches java objects.	The distributed cache is a component that allows developers to deploy jars for Map-Reduce processing.

19. What is the best performance one can expect from a Hadoop cluster?
The best performance expectation one can have is measured in seconds. This is because Hadoop can only be used for batch processing	The best performance expectation one can have is measured in milliseconds. This is because Hadoop executes in parallel across so many machines
The best performance expectation one can have is measured in minutes. This is because Hadoop can only be used for batch processing	It depends on on the design of the map-reduce program, how many machines in the cluster, and the amount of data being retrieved

20. What is writable?
Writable is a java interface that needs to be implemented for streaming data to remote servers.	Writable is a java interface that needs to be implemented for HDFS writes.
Writable is a java interface that needs to be implemented for MapReduce processing.	None of these answers are correct.