SUMMARY

• Around 9 years of Information Technology experience.
• Extensive experience in administration, design, development and implementations of robust technology systems, with expertise in Data Warehousing solutions and BIGDATA technologies.
• Able to understand business and technical requirements quickly; Excellent communications skills and work ethics; Able to work independently; Experience working with clients in the Retail and Banking industries.
• Around 9 years of experience in Hadoop, Linux Administration and ETL DataStage
• Worked exclusively on MapR and Cloudera Distribution of Hadoop.
• Hands on experience in Installation, Configuration of HADOOP cluster, cluster maintenance, monitoring and troubleshooting.
• Installation and configuration of ElasticSearch and Synthesys(for ETL processing), Postfix(SMTP) in Linux environment.
• Hands on experience in deploying and managing the multi - node Hadoop cluster with different Hadoop ecosystems HIVE, SQOOP, HBASE, PIG.
• Good hands on experience on LINUX Adminstration and troubleshooting issues related to Network and OS level.
• Assisted in designing, developing and architecture of HADOOP ecosystem.
• Experience in capacity planning and nodes forecasting.
• Worked for Name node recovery and balancing Hadoop Cluster..
• Strong experience in System Administration, Upgrading, Patches, Troubleshooting, Security, Backup, Disaster Recovery.
• Experience on Cloudera Hadoop upgrade from CDH4.3 to CDH5.3 and applying patches.
• Worked on setting up Name Node High Availability for major production cluster and designed automatic failover control using zookeeper and quorum journal nodes.
• Configured Oozie workflow scheduler in clusters.
• Configured ZooKeeper to implement node coordination in clustering support.
• Configuration of Postfix (i.e SMTP) for message transfer.
• Experience on Commissioning, Decommissioning, Balancer and Managing Nodes and tuning server for optimal performance of the cluster.
• In depth knowledge and good understanding of Hadoop daemons: Name Node, Data Node, Secondary Name Node, Resource Manager, Node Manager.
• Worked on Kafka for the proof of concept for carrying out log processing on distributed system.
• Installed Apache Kafka on Dev, UAT and Prod Environment.
• Worked on kafka tasks like creation/deletion/modification of topic.
• Expereienced on Kafka broker recovery, partition reassignments and other admin functions.
• Installation and configuration of Sqoop and Flume,Flume agents
• Worked on Flume to collect, aggregate, and store the web log data from different sources
• Expereinced in importing and exporting data using Sqoop from RDBMS(Oracle/MySQL Server) to HDFS.
• Using Splunk, performed Log analaysis, alerts etup, reporting (Dashboard) health of Hadoop Servers as part of production system montoring.
• Experience in dealing with structured, semi-structured and unstructured data in HADOOP ecosystem.
• Optimizing performance of Hbase/Hive/Pig/Sqoop jobs.
• Experienced on CHEF Automation and Deployment for Hadoop cluster
• Configured Fair scheduler schedulers in cluster. Configured capacity scheduler as part of POC.
• Performance Monitoring and Fine-Tuning on Linux systems.
• Used Crontab for automating montoirng CPU consumption, Memory Utilizaion, Disk Utilization jobs
• Experience monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network
• Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
• Experience in handling security for Hadoop Cluster with Kerberos.
• Provided On Call Support to fix the various Production issues on the fly in both Hadoop and ETL Projects to provide smooth running of jobs in peak time.
• Created engineering Documentation to be used by the operations counterparts and create high-quality artifacts required by change control processes, security and the project team.
• Excellent interpersonal and communication skills, technically competent and result-oriented to problem solving and leadership skills.

TECHNICAL SKILLS

Hadoop Tools: MaprR 5.2.0, CDH5.3 and CDH4.3, HDFS,YARN, MapReduce,Kafka,HBase, Hive, Flume, Sqoop, Zookeeper,Oozie,Splunk Fair Scheduler,CHEF Automation,Kerberos, LDAP, ElasticSearch

ETL Tools: IBM DataStage 8.5 and 7.5, Data Quality 9.1, Platform LSF 9.1.1, Grid Tool Kit 5.0, IBM MFT, IBM WMQ and Control-M Scheduler.

Other Utility Tools: Winscp, Putty

Operating System: RedHat Linux RHEL 6.x/5.x, UNIX, CentOS, Windows (98/XP/Vista/2008/2010)

Databases: MySQL, SQL Server, Oracle 9i and 10g, NoSQL Databases HBase, DB2,Teradata

Scripting: Bash Shell Scripting

Programming: C, Core Java, Basic Map Reduce, SQL, HIVE, PIG

PROFESSIONAL EXPERIENCE

Hadoop Administrator

Confidential, Minneapolis

Responsibilities:

• Installation, configuration, deployment of MapR 5.2.0 Hadoop cluster in environments such as Development,UAT and Production.
• Installation, Configuration of ElasticSearch-2.4.4 for Log data analysis and Synthesys for ETL operations.
• Configuration and testing of Postfix (SMTP) for Message transfer from Source Systems to Edge Nodes (Local).
• Commissioning and Decommissioning Nodes from time to time.
• Configuration of Oozie Workflow Scheduler in MapR cluster and tested sample jobs.
• Worked on Data capacity planning and nodes forecasting.
• Responsible for deciding the hardware configurations for the cluster .
• Worked for Name node recovery and Balancing Hadoop Cluster.
• Implemented and tested Fair Scheduler in MapR cluster to check sharing of resources for the Map Reduce and Spark jobs run by the users.
• Implemented LDAP interation with Hadoop to secure user access on hadoop cluster nodes.
• Implemented automatic failover zookeeper and zookeeper failover controller as part of High Availability.
• Carried out heavy logs analysis using Elastci Search.
• Monitoring cluster jobs on daily basis, tuning system performance related configuration parameters, backing up configuration xml files in case of changes.
• Monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Experienced in managing and reviewing Hadoop log files.
• Good experience with Hadoop Ecosystems such as Hive, HBase, Pig and Sqoop.
• Import data using Sqoop to load data from Oracle/SQL Server/MySQL Server to HDFS on regular basis.
• Production support responsibilities include entire cluster maintenance and make sure cluster is up and run all times.
• Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.
• Worked with network and system engineers to define optimum network configurations, server hardware and operating system.
• Documented the systems processes and procedures for future references.
• Provided On Call Support to fix the various Production issues on the fly to provide smooth running of jobs in peak time.

Environment: s/Tools: MapR 5.2.0, HDFS, YARN,Hive,HBase,Sqoop, Zookeeper, Oozie 4.2.0, Fair Scheduler, Oracle DB, ElasticSearch 2.4.4, Synthesys ETL 4.1.3, Autosys, Core Java, Linux, Bash scripts.

Hadoop Administrator

Confidential, Minneapolis

Responsibilities:

• Experience in Installation, configuration, deployment, maintenance, monitoring and troubleshooting Cloudera Hadoop clusters in environments such as Development and Production.
• Commissioning and Decommissioning of Nodes from time to time in cluster.
• Solo responsible for everything related to clusters starting from maintaining, monitoring and keeping up the cluster all the time by supporting 24/7 to support business without any outages
• Worked for Name node recovery and Balancing Hadoop Cluster .
• Experienced in Cloudera Hadoop upgrade from CDH4.3 to CDH5.3 and applying patches
• Worked on Data capacity planning, nodes forecasting and determining the correct hardware and infrastructure for cluster.
• Responsible for managing and scheduling jobs on a Hadoop Cluster.
• Configuration of Oozie Workflow Scheduler and Testing sample jobs.
• Monitoring all daemons, cluster health status on daily basis and tuning system performance related configuration parameters, backing up configuration xml files.
• Implemented Fair Scheduler to share the resources of the cluster for the Map Reduce jobs run by the users.
• Good experience with Hadoop Ecosystem components such as Hive, HBase, Pig, Sqoop and Optimizing performance of Jobs.
• Worked on Kafka for the proof of concept for carrying out log processing on distributed system.
• Installed Apache Kafka on Dev, UAT and Prod Environment.
• Populated HDFS with huge amounts of data using Apache Kafka.
• Worked on kafka tasks like creation/deletion/modification of topic.
• Worked on Kafka broker recovery, partition reassignments manually and other admin functions
• Installation and configuration of Sqoop and Flume,Flume agents.
• Worked on Flume to collect, aggregate, and store the web log data from different sources.
• Import data using Sqoop to load data from Oracle Server/MySQL Server to HDFS on regular basis.
• Troubleshooting, Manage and review data backups, Manage and review Hadoop log files.
• Using Splunk, performed Log analaysis, alerts etup, reporting (Dashboard) health of Hadoop Servers as part of production system montoring.
• Installed splunk sgent in Hadoop Servers.
• Assisted with Automatic Deployment using CHEF.
• Implemented security for Hadoop Cluster with Kerberos.
• Expereinced in LDAP integration with hadoop and access provisioning for secured cluster.
• Used crontab for automating monitoring jobs for alerting on CPU consumption, Memory Utilizaion, Disk Utilization.
• Work with Hadoop developers, designers in troubleshooting Map Reduce job failures and issues.
• Work with network and system engineers to define optimum network configurations, server hardware and operating system.
• Provided On Call Support to fix the various Production issues on the fly to provide smooth running of jobs in peak time.

Environment: s/Tools: CDH5.3 and CDH4.3,Apache Hadoop 2.x, HDFS, YARN, Map Reduce, Kafka,Hive, HBase, Pig, Sqoop,Flume, Zookeeper,Oozie,Fair Scheduler, Splunk,LDAP,Kerberos, Oracle Server, MySQL Server, ElasticSearch, CHEF Automation, Crontab,Core Java, Linux, Bash scripts

Linux Administrator

Confidential, Minneapolis

Responsibilities:

• Installation and Administration of Redhat Linux server.
• Installation and configuration of Apache Hadoop 0.x/1.x and other Hadoop ecosystems like Hive, HBase and Sqoop as part of POC.
• Experienced in Recovery of NameNode failure..
• Scheduling and Monitoring Jobs on Hadoop cluster.
• Worked on importing and exporting data from Oracle and MySql into HDFS and HIVE using Sqoop.
• Implementing with Customer recommended Latest Patches, Packages and Impact Analyzing.
• Performed Kernel tuning with the sysctl and installed packages with yum and rpm.
• Performed Disk management with the help of LVM (Logical Volume Manager) in Linux.
• Troubleshooting Network File System (NFS) issues.
• Performing Installation/Upgrade, System Startup and System Shutdown as needed.
• Perform system boot procedures.
• Experience in troubleshooting High Availability Solutions using Red hat Cluster Server in Linux.
• Performed Kernel tuning with the sysctl.
• Managing disks and file system Logical Volume Manager in Linux servers.
• Backup management Recovery through Legato.
• Management of Red Hat Linux user accounts, groups, directories and file permissions.
• Maintaining system logs and ensuring system security and integrity.
• Automating jobs by using Cron.
• Troubleshooting for filesystem crash, disk space problems, and network problems.
• Monitoring Virtual memory, adding additional swap space as needed.
• Performance monitoring using SAR, Iostat, VMstat and MPstat on servers.
• File System administration and management like monitoring various logs, file system sizes etc.
• User creation/administration using LDAP.
• Used crontab for automating monitoring jobs for alerting on CPU consumption, Memory Utilizaion, Disk Utilization.
• Maintaining and changing file systems and disk assignments using Meta disk.
• Established and maintained network users, user environment, directories, and security
• Coordinating with various back end support teams like Redhat for Root Cause Analysis (RCA).
• Provided 24/7 on call support on Production Servers.

Environment: s/Tools: RHEL 5.x, Logical Volume Manager, Apache Hadoop 0.20.2/Hadoop 1.2.0, HDFS, Hive, Sqoop, LDAP,Bash Scritping.

DataStage Administrator

Confidential, Minneapolis

Responsibilities:

• Installation and configuration of Information Server version 8.5 and DataQuality 9.1
• Installation and configuration of Grid toolkit and Load Leveler for Dynamic resource management.
• Apply fix packs, patches and service packs as required to keep the DataStage environment up-to-date.
• Used ISTOOL to migrate job objects between different environments in IS Manager.
• Handled issues occured in LSF Grid and SSH setup and issues.
• Configuration of dsenv, uvconfig, DSParams, uvodbc.config files at both engine and project levels.
• Managed Users, Groups, active sessions, logs, schedules and licenses in IIS Web Console.
• Fine tune the environment to avoid any bottle necks and performance.
• Monitoring and managing CPU Utilization, Peak Disk Utilization,File systems and Network load on each server through Tivoli Performance Viewer.
• Provided On Call Support to fix the various Production issues on the fly to provide smooth running of ETL jobs in peak time.
• Raised PMRs with IBM on up gradation problems, patches and fix packs.

Environment: s/Tools: IBM InfoSphere Information Server 8.5/DataStage 7.5/Suite, Oracle 9i and 10g, SQL Server, Teradata, IBM Managed File Transfer(MFT), IBM Websphere Message Queues(WMQ), Shell Scripts, Control M Scheduler.

DataStage Developer

Confidential, Minneapolis

Responsibilities:

• Worked in development phase meetings for Business Analysis and Requirements Gathering.
• Designed, developed and tested ETL Process for EIW Integration.
• Design and development of Star Join Schema.
• Used DataStage Designer to design and develop jobs for extracting, cleansing, transforming, integrating, and loading data into different Data Marts.
• Also worked with DataStage Administrator to create Repository, User groups, Users and managed users by setting up their privileges and profiles.
• Designed and developed jobs using Parallel Extender for splitting bulk data into subsets and to dynamically distribute to all available nodes to achieve best Job performance.
• Created Batch Processes for running the jobs in parallel.
• Involved in migration of DataStage jobs from development to production environment.
• Created DataStage Server jobs to load data from sequential files, flat files and Oracle tables.
• Handled several External Database extractions from Oracle, Netezza, EXCEL file transformations.
• Defined the data definitions and created the Confidential tables in the database.
• Performed Unit Testing and tuned for better performance with updates on data warehouse tables using DataStage Director for jobs Monitoring and troubles shooting.

Environment: IBM DataStage 7.5, Oracle 10g, SQL Server 2008, Shell Scripts, Control M Scheduler.