index에 문제가 생겨 해당 네임으로 다시 인덱스 생성(mapping 부터) 할 시에 이전 index가 계속 나오는 현상.

index delete 후에도 index 가 계속 나옴. 업데이트 됐다고 하는데 일부버전에서 문제가 계속 되나 보다.

로컬 gateway 의 recovery에서 dangled index 를 계속 올리는 현상인듯 하다.

관련 url )  https://github.com/elastic/elasticsearch/issues/2067

1) index delete 

$ curl -XDELETE 'http://localhost:9200/indexname/'

해당 해결 방법

elasticsearch.yml 에 

gateway.local.auto_import_dangled : no 

설정을 주고 es 재시작 후 index 생성한다. 

http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#logs

Dump the container logs

  Usage: yarn logs <options>

example) yarn logs -applicationId job1238547128312

참고 자료 :http://lucene.apache.org/core/2_9_4/api/core/org/apache/lucene/search/Similarity.html

• 요소항목 설명

 -Term Frequency(tf) : 쿼리하는 키워드가 몇번 나왔는지에 대한 수치이다. 나온 수치의 루트값을 이용한다. 
        tf(t in d) = √frequency
    -Inverse Document Frequency(idf) : 역빈도 내용인다. 전체 document에서 해당 term이 얼마나 나왔는지를 측정. 다수의 문서에서 나온다면 이 term은 중요하지 않은 것을 간주하고 낮은 weight를 갖는다.
        idf(t) = 1 + log ( numDocs / (docFreq + 1)) 
	-field length norm : term의 중요도를 측정하는 수치이다. 해당 document에 term의 갯수가 적을 때 더 큰 weight를 갖는다. term의 갯수의 루트 값을 이용한다. 
		ex) query string : 로이조, 로이조 주말 핫녹(high), 로이조 주말 핫 녹화 방송(low) <둘을 대상으로 했을때 상대적인>        
		norm(d) = 1 / √numTerms
    -queryNorm : 두개 이상의 term으로 쿼리할때 두개의 쿼리에 대한 normalize를 위함이다. 두개의 term으로 쿼리할 경우 query:{ bool:{should:[{term:{field:'aaa'}},{term:{field:'bbb'}}] } 로 
                 검색하게 된다. 이 과정에서 두개의 쿼리로 쪼개지고 두 단어에 대한 상대적인 차이(관계)를 구하기 위함이다. term이 1개일 경우 의미 없다. 
                 =1 / √sumOfSquaredWeights    /     sumOfSqueredWeights = Sigma(t in q)(idf(t) * getBoost(t in q))^2 // 

    -coord : 검색 term이 많이 hit될 수록 보상하는 수치. 검색된 문서에서 쿼리의 Term이 몇 개 들어있는지에 대한 값. (3개단어로 검색쿼리 했고 2개 단어가 맞아 떨어짐. 해당 단어에 대한 점수  * 2/3)
    -t.getboost : 쿼리시에 넣을 수 있는데 더 높은 가중치를 줘서 검색하게 됨. queryNorm 계산시에 들어감. 
 

• 계산 되는 순서 및 로직

 1) 각 term단위로 fieldweight(tf * idf * fieldnorm) 을 구한다.
 2) 각 term단위로 queryNorm 을 구한다. 
 3) 위 둘을 곱한 값들을 갖고 더한다. 

• multi term에서 유사관계 판단

cosine similarity 관계. happy hippopotanus로 쿼리 하였을 때 happy 2, hippopotamus 5 의 weight를 갖는다고 하자. 그럼 happy 만 들어간 doc, hippopotamus 만 들어간 문서, 둘 다 들어간 문서에서 각 단어의 weight를 구해 아래와 같이 표현한다. 그리고 쿼리의 벡터와 비슷한 각도를 따지는 것이다. 아래에서는 3번의 doc이 일치했다. happy와 같은 단어는 common word 이므로 어디서나? 낮은 점수가 나올 것 이다.

ex)1) you look happy, 2) you look like hippopotamus 3)you look so happy, hippopotanus 

git show-ref -h HEAD 

1) 현재 해드의 커밋 포인트 보기

git fsck --lost-found

dangling commit 7c61179cbe51c050c5520b4399f7b14eec943754

git reflog
39ba87b... HEAD@{0}: HEAD~1: updating HEAD
7c61179... HEAD@{1}: pull origin master: Fast forward

2) fsck 명령으로 reset한 커밋 포인트의 해쉬를 알 수 있다.\

git merge 7c61179

3) 해당 commit 포인트로 merge 하면 reset한 commit 포인트로 복구 된다.

You might be familiar with Linux load averages already. Load averages are the three numbers shown with the uptime and top commands - they look like this:

Most people have an inkling of what the load averages mean: the three numbers represent averages over progressively longer periods of time (one, five, and fifteen minute averages), and that lower numbers are better. Higher numbers represent a problem or an overloaded machine. But, what's the the threshold? What constitutes "good" and "bad" load average values? When should you be concerned over a load average value, and when should you scramble to fix it ASAP?

First, a little background on what the load average values mean. We'll start out with the simplest case: a machine with one single-core processor.

The traffic analogy

A single-core CPU is like a single lane of traffic. Imagine you are a bridge operator ... sometimes your bridge is so busy there are cars lined up to cross. You want to let folks know how traffic is moving on your bridge. A decent metric would be how many cars are waiting at a particular time. If no cars are waiting, incoming drivers know they can drive across right away. If cars are backed up, drivers know they're in for delays.

So, Bridge Operator, what numbering system are you going to use? How about:

• 0.00 means there's no traffic on the bridge at all. In fact, between 0.00 and 1.00 means there's no backup, and an arriving car will just go right on.
• 1.00 means the bridge is exactly at capacity. All is still good, but if traffic gets a little heavier, things are going to slow down.
• over 1.00 means there's backup. How much? Well, 2.00 means that there are two lanes worth of cars total -- one lane's worth on the bridge, and one lane's worth waiting. 3.00 means there are three lane's worth total -- one lane's worth on the bridge, and two lanes' worth waiting. Etc.

 = load of 1.00

 = load of 0.50

 = load of 1.70

This is basically what CPU load is. "Cars" are processes using a slice of CPU time ("crossing the bridge") or queued up to use the CPU. Unix refers to this as the run-queue length: the sum of the number of processes that are currently running plus the number that are waiting (queued) to run.

Like the bridge operator, you'd like your cars/processes to never be waiting. So, your CPU load should ideally stay below 1.00. Also like the bridge operator, you are still ok if you get some temporary spikes above 1.00 ... but when you're consistently above 1.00, you need to worry.

So you're saying the ideal load is 1.00?

Well, not exactly. The problem with a load of 1.00 is that you have no headroom. In practice, many sysadmins will draw a line at 0.70:

• The "Need to Look into it" Rule of Thumb: 0.70 If your load average is staying above > 0.70, it's time to investigate before things get worse.

• The "Fix this now" Rule of Thumb: 1.00. If your load average stays above 1.00, find the problem and fix it now. Otherwise, you're going to get woken up in the middle of the night, and it's not going to be fun.

• The "Arrgh, it's 3AM WTF?" Rule of Thumb: 5.0. If your load average is above 5.00, you could be in serious trouble, your box is either hanging or slowing way down, and this will (inexplicably) happen in the worst possible time like in the middle of the night or when you're presenting at a conference. Don't let it get there.

What about Multi-processors? My load says 3.00, but things are running fine!

Got a quad-processor system? It's still healthy with a load of 3.00.

On multi-processor system, the load is relative to the number of processor cores available. The "100% utilization" mark is 1.00 on a single-core system, 2.00, on a dual-core, 4.00 on a quad-core, etc.

If we go back to the bridge analogy, the "1.00" really means "one lane's worth of traffic". On a one-lane bridge, that means it's filled up. On a two-late bridge, a load of 1.00 means its at 50% capacity -- only one lane is full, so there's another whole lane that can be filled.

 = load of 2.00 on two-lane road

Same with CPUs: a load of 1.00 is 100% CPU utilization on single-core box. On a dual-core box, a load of 2.00 is 100% CPU utilization.

Multicore vs. multiprocessor

While we're on the topic, let's talk about multicore vs. multiprocessor. For performance purposes, is a machine with a single dual-core processor basically equivalent to a machine with two processors with one core each? Yes. Roughly. There are lots of subtleties here concerning amount of cache, frequency of process hand-offs between processors, etc. Despite those finer points, for the purposes of sizing up the CPU load value, the total number of cores is what matters, regardless of how many physical processors those cores are spread across.

Which leads us to a two new Rules of Thumb:

• The "number of cores = max load" Rule of Thumb: on a multicore system, your load should not exceed the number of cores available.

• The "cores is cores" Rule of Thumb: How the cores are spread out over CPUs doesn't matter. Two quad-cores == four dual-cores == eight single-cores. It's all eight cores for these purposes.

Bringing It Home

Let's take a look at the load averages output from uptime:

This is on a dual-core CPU, so we've got lots of headroom. I won't even think about it until load gets and stays above 1.7 or so.

Now, what about those three numbers? 0.65 is the average over the last minute, 0.42 is the average over the last five minutes, and 0.36 is the average over the last 15 minutes. Which brings us to the question:

Which average should I be observing? One, five, or 15 minute?

For the numbers we've talked about (1.00 = fix it now, etc), you should be looking at the five or 15-minute averages. Frankly, if your box spikes above 1.0 on the one-minute average, you're still fine. It's when the 15-minute average goes north of 1.0 and stays there that you need to snap to. (obviously, as we've learned, adjust these numbers to the number of processor cores your system has).

So # of cores is important to interpreting load averages ... how do I know how many cores my system has?

cat /proc/cpuinfo to get info on each processor in your system. Note: not available on OSX, Google for alternatives. To get just a count, run it through grepand word count: grep 'model name' /proc/cpuinfo | wc -l

http://nginx.org/en/docs/http/ngx_http_stub_status_module.html

참조 : http://egloos.zum.com/sweeper/v/3002220

자바에서 기본 스트링 인코딩을 utf-16을 쓴다 16비트냐 32비트냐 인데 utf-8은 표현 메모리?구간 마다 길이가 가변적이다.

한글 완성형의 코드 포인트 범위는 U+AC00~U+D7AF이므로, UTF-8 인코딩에서 한글은 무조건 3바이트 인코딩이다. 그래서 URL에 파라미터 값이 %ED%95%9C%EA%B8%80과 같이 표시된다면 UTF-8 인코딩일 확률이 높다.한글의 모든 글자들은 3byte 구간에 있으므로 utf-8로 변환할 경우 char='한'은 3byte가 된다.

참조 http://helloworld.naver.com/helloworld/76650

for i in {1..5}
do
   echo "$i"
done