目的:
为impala jdbc提供统一的接口,作用参照http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_proxy.html
步骤:
安装haproxy
选择一台非impalad的机器安装haproxy
yum install haproxy
编辑/etc/haproxy/haproxy.cfg,参考
global
# To have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
#stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#
# You might need to adjust timing values to prevent timeouts.
#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
#option forwardfor except 127.0.0.0/8
option redispatch
retries 3
maxconn 3000
timeout connect 5000
timeout client 50000
timeout server 50000
#
# This sets up the admin page for HA Proxy at port 25002.
#
listen stats :25002
balance
mode http
stats enable
stats auth username:password
# This is the setup for Impala. Impala client connect to load_balancer_host:25003.
# HAProxy will balance connections among the list of servers listed below.
# The list of Impalad is listening at port 21000 for beeswax (impala-shell) or original ODBC driver.
# For JDBC or ODBC version 2.x driver, use port 21050 instead of 21000.
listen impala :25003
mode tcp
option tcplog
balance leastconn
server cdhslave1 cdhslave1.yeahmobi.com:21050 check
server cdhslave2 cdhslave2.yeahmobi.com:21050 check
server cdhslave3 cdhslave3.yeahmobi.com:21050 check
启动haproxy
haproxy -f /etc/haproxy/haproxy.cfg
关闭service haproxy stop
impala配置
impala daemon group->advanced->Impala Daemon Command Line Argument Advanced Configuration Snippet (Safety Valve)
-principal=impala/cdhmaster.yeahmobi.com@YEAHMOBI.COM
-be_principal=impala/_HOST@YEAHMOBI.COM
测试
private static final String IMPALAD_HOST = "cdhmaster.yeahmobi.com";//haproxy server的hostname
private static final String IMPALAD_JDBC_PORT = "25003";//端口选择haproxy的代理端口
private static final String CONNECTION_URL = "jdbc:hive2://" + IMPALAD_HOST + ':' + IMPALAD_JDBC_PORT + "/data_system;user=hive;password=111111";//ldap用户及密码
使用时发现,impala-shell不好使了,不能正常连接,去除如上粗体步骤后,impala-shell正常,jdbc HAproxy也正常使用。
分享到:
相关推荐
springboot集成impala,包含yml文件、impala配置Bean、ImpalaJDBC41-2.6.4.1005.jar和impala在pom.xml中的配置,有问题留言
Apache Impala Guide impala-3.3
由于Impala的版本是CDH的,所以安装Impala的涉及到的Hive、Hadoop、Spark、Hbase等环境也要统一使用CDH版本。 本文档部署impala时所用到的Hive、Hadoop等环境的安装方式是使用tar包方式安装的(cdh版本的tar包安装...
BIEE配置ODBC连接impala BIEE配置ODBC连接impala BIEE配置ODBC连接impala
从零开始讲解大数据分布式计算的发展及Impala的应用场景,对比Hive、MapReduce、Spark等类似框架讲解内存式计算原理,基于Impala构建高性能交互式SQL分析平台 课程亮点 1,知识体系完备,从小白到大神各阶段读者均...
dbeaver中连接impala所需jdbc包,适用于CDH5.16.2,impala2.12
impala的使用教程impala的使用教程impala的使用教程impala的使用教程impala的使用教程
用于JDBC连接Impala的工具包
impala的驱动下载,com.cloudera.impala.jdbc41.Driver
详细介绍了impala创建自定义函数去除‘-’的uuid,包括maven所用的pom.xml
springboot+mybatis+impala/mysql整合Demo , 内嵌PageHelper插件已整合,需要根据pom.xml中的备注操作即可使用mysql和PageHelper, impala 不支持PageHelper插件
Impala provides fast, interactive SQL queries directly on your Apache Hadoop data stored in HDFS, HBase, or theAmazon Simple Storage Service (S3). In addition to using the same unified storage ...
分析了Cloudera公司推出的Impala实时查询引擎原理与架构,并深入比较Impala与传统MapReduce的性能与特点,针对Impala进行复杂大数据处理方面的不足,提出了MapReduce与Impala结合的大数据处理方法,通过使用 ...
Impala 驱动包 Cloudera_ImpalaJDBC4_2.5.41.zip Cloudera_ImpalaJDBC41_2.5.41.zip Cloudera-JDBC-Driver-for-Impala-Install-Guide.pdf Cloudera-JDBC-Driver-for-Impala-Release-Notes.pdf
impala cookbook详解版
Impala权威安装文档
由于impala处理日期的函数如date_sub(),date_trunc(),last_day()等这些日期处理函数还需要进行日期格式化为yyyy-MM-dd使用,sql代码段过长,导致频繁嵌套过于复杂.所以自定义udf函数解决这些问题.以下为实现过程.
impala的substr()和substring()函数是不支持中文的,创建一个udf解决impala sql中substr()函数截取中文字符串乱码的问题
impala安装与添加
Impala的ODBC去驱动,下载自cloudera官网,留做备份。。。。。。。。。。